Results 1 to 2 of 2
  1. #1
    Join Date
    Jan 2012

    Unanswered: MySQL Performance Help

    Dear All:
    I am quite new to MySQL and have a marathon task of making a huge database run fast in all ways

    The system:
    MySQL 5.1
    The database goes through ~50k row insertion per second. The average row size is 380. All the columns are big int unsigned not null.
    The available memory is 4 GB.
    Disk size 1.2 TB RAID 1.
    The table is partitioned for every 1 hour, example: YYYYMMDDHH. There are not indexes on the table except for the large composite key with 5 columns starting with time stamp column as the first column in the key.
    Engine: InnoDB only

    The challenge:
    The data that is getting inserted every seconds needs to be aggregated at the end of 1 hour for hourly summarization.
    The hourly summarized data needs to be aggregated again at the end of the day for daily summarization.
    There are set of queries(procedures) that can access data for real time analytics. The duration can be for last 10 minutes, last 1 hour and sometimes for a given time interval which is the past (Custom reports).

    Current status:
    Insertion speed seems to be good with around 2 seconds for each file.
    Query performance for 5 minutes seems to be just fine. Anything beyond this is completely not in the acceptable range.

    Below is the My.cnf file snapshot:

    # Optimized mysql configuration file with MySQL support inputs

    # The following options will be passed to all MySQL clients
    port = 3306
    socket = /tmp/mysql.sock

    # Here follows entries for some specific programs

    # The MySQL server
    port = 3306
    socket = /tmp/mysql.sock
    key_buffer_size = 256M
    max_allowed_packet = 1M
    table_open_cache = 1024
    sort_buffer_size = 1024M
    read_buffer_size = 256M
    read_rnd_buffer_size = 64M
    thread_cache_size = 8
    #query_cache_size= 32M
    join_buffer_size = 12M
    # Temp directory changed
    tmpdir = /usr/local/var/tmp
    # Maximum # of connections
    max_connections = 51
    # Max heap table size
    max_heap_table_size = 1024M
    query_cache_size= 0
    transaction-isolation = READ-UNCOMMITTED

    # Set the open_files_limit to a higher number(partition with the innodb_file_per_table
    # option enabled requires more open files
    open_files_limit = 32768

    # Don't listen on a TCP/IP port at all. This can be a security enhancement,
    # if all processes that need to connect to mysqld run on the same host.
    # All interaction with mysqld must be made via Unix sockets or named pipes.
    # Note that using this option without enabling named pipes on Windows
    # (via the "enable-named-pipe" option) will render mysqld useless!

    # Disable Federated by default

    # Replication Master Server (default)
    # binary logging is required for replication

    # binary logging format - mixed recommended
    # number of days to keep binary logs

    # required unique id between 1 and 2^32 - 1
    # defaults to 1 if master-host is not set
    # but will not function as a master if omitted
    server-id = 1

    # Try number of CPU's*2 for thread_concurrency
    innodb_thread_concurrency = 32
    innodb_table_locks = 0
    innodb_file_per_table = 1
    innodb_data_home_dir = /usr/local/var/
    innodb_buffer_pool_size = 4G
    # Set .._log_file_size to 25 % of buffer pool size
    innodb_additional_mem_pool_size = 32M
    innodb_log_buffer_size = 16M
    innodb_lock_wait_timeout = 120
    innodb_open_files = 32768
    innodb_log_file_size = 512M
    innodb_doublewrite = 0
    innodb_file_io_threads = 16
    innodb_flush_log_at_trx_commit = 2
    innodb_flush_method = O_DIRECT
    innodb_checksums = 0

    max_allowed_packet = 16M

    # Remove the next comment character if you are not familiar with SQL

    key_buffer = 128M
    sort_buffer_size = 128M
    read_buffer = 2M
    write_buffer = 2M

    key_buffer = 128M
    sort_buffer_size = 128M
    read_buffer = 2M
    write_buffer = 2M


    Thanks for the help,
    Arun Shankar

  2. #2
    Join Date
    Jan 2012

    Can you provide the slow logs, the explain of the slow quires, and the table description of the tables in the slow query if possible.

    Because your server only has 4G memory,
    you can try change these parameters:

    sort_buffer_size = 1024M   To 1M
    read_buffer_size = 256M    To 1M
    read_rnd_buffer_size = 64M  to 1M
    thread_cache_size = 8
    #query_cache_size= 32M
    join_buffer_size = 12M      to 1M or less

    innodb_buffer_pool_size = 4G to 2.4G or less

    good luck

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts