MarkhamDBA, I believe BUFFPAGE is or has been deprecated and shouldn't be used. Stick with the number of pages.
In V9.5 you can use the Automatic parameter with SIZE and DB2 will adjust the size according to workload.
If you can't use this, I don't know of a formula but you can make sum educated guesses.
How are the large tables accessed? If it is a tablespace scan, you need the bufferpool large enough to handle all the pages (if you want them left in the bufferpool). If it is Index access, then only a few pages (index and table) will be read into the bufferpool (regardless of the table size).
How many of the large tables will be accessed at the same time (assumming table space or partition scanning)? If one at a time, then the size of the largest table space would be a starting point.
Basically how many table and/or index pages do you want left in the bufferpool to reduce I/O?
Also keep in mind the various thresholds which will initiate taskes to remove pages.
PS Another scheme you might think sounds the same but is a little backward from your approach. That is to split out your Small tables (especially code type tables) into a separate bufferpool sized so that all the pages of all the tables will say resident. Since they are code tables, there will be little if any Changes made to the pages and DB2 will not be in any hurry to remove the pages. Then all your other data (medium and large) can be in one (or two) other 'as large as you can make them' bufferpools(s).