We are using MySQL 5.044 and have the following replication topology: 1 Master and 3 identical slaves.
Master machine has 7GB of memory and Innodb buffer is set to 2 GB.
Each slave has 35 GB of memory and innodb buffer is set to 25 GB.
Recently we began loading a lot of data on a daily basis (~ 300,000 articles) and I noticed that throughout the day the memory on the machine which starts out with ~ 31GB after mysqld service starts ends up with just 16MB of memory when I run "free -m" command.
The data load involves a lot if inserts, but also deletes and updates, so my initial assumption was that a lot of data goes into the innodb buffer pool (due to selects during insert/delete/update process?) and the buffer is not cleared up after data is commited to the database. However, doing more investigation seems to indicate that this is not the case (maybe only partially the case). Namely, I just looked at how much memory is used up now using "free -m" command, and it says 15GB. I then look at how much innodb buffer pool is used up using 'mysqlreport tool' and it shows that only 2.79GB are used.
So my questions are:
1. Where is the other 12GB going?
2. How does innodb buffer flush/clear up its contents? What is the internal algorithm?
3. Any other suggestions I can use to investigate the memory usage? [i.e. once i totally run out of memory again on the machine, what can I look at to get an indication of what the memory is being used for?]
Below is the .cnf file for the machine in questions: