Results 1 to 3 of 3
  1. #1
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367

    Unanswered: how to check for pinned memory on Linux RHEL

    db2 is getting ENOMEM (out of memory) from pwrite() system call, but there is enough free memory as per free/vmstat. Linux system message log doesn't contain any memory related errors.


    From db2diag.log:

    FUNCTION: DB2 UDB, oper system services, sqloseekwrite64, probe:40
    MESSAGE : ZRC=0x870F0024=-2029060060=SQLO_MEM "out of memory"
    DIA8533C The system memory limit was reached.
    CALLED : OS, -, pwrite OSERR: ENOMEM (12)

    ...

    FUNCTION: DB2 UDB, buffer pool services, sqlbWriteBlocks, probe:20
    MESSAGE : ZRC=0x870F0024=-2029060060=SQLO_MEM "out of memory"
    DIA8533C The system memory limit was reached.
    DATA #1 : String, 35 bytes
    No more memory available in system.

    ....

    FUNCTION: DB2 UDB, buffer pool services, sqlbWritePage, probe:20
    MESSAGE : ZRC=0x85020005=-2063466491=SQLB_NO_MEMORY
    "No pageread because no-mem error from OSS"
    DIA8305C Memory allocation failure occurred.




    Memory Usage Info, '/usr/bin/free'

    total used free shared buffers cached
    Mem: 16433336 9566916 6866420 0 388560 8411152
    -/+ buffers/cache: 767204 15666132
    Swap: 1020088 0 1020088


    Memory Usage Info, '/usr/bin/vmstat 1 5'

    procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
    r b swpd free buff cache si so bi bo in cs us sy id wa st
    0 0 0 6866428 388560 8411108 0 0 1 23 1 1 0 1 99 0 0
    0 0 0 6866428 388560 8411148 0 0 0 46 1188 978 0 0 100 0 0
    0 0 0 6866428 388560 8411148 0 0 0 66 1081 964 0 0 100 0 0
    0 0 0 6866492 388560 8411152 0 0 0 117 1117 941 0 0 100 0 0
    0 0 0 6866420 388560 8411152 0 0 8 53 1076 962 0 0 100 0 0



    So, it could that some memory is pinned at the OS level??? On AIX, I can check for v_pinshm and use vmstat -v / svmon to check amount of pinned memory. Does anyone know how to check for the same on Linux?


    RHEL 5.2 , x86_64

  2. #2
    Join Date
    May 2003
    Location
    USA
    Posts
    5,737
    I would make sure your Linux kernel parms are set correctly per the following output:
    ipcs l
    Check the online doc for recommendations on setting kernel parms, depending on which DB2 version you have.

    I would also recommend that you consider turning off STMM, or at least hard-coding memory for bufferpools, LOCKLIST, MAXLOCKS, and SORTHEAP.
    M. A. Feldman
    IBM Certified DBA on DB2 for Linux, UNIX, and Windows
    IBM Certified DBA on DB2 for z/OS and OS/390

  3. #3
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    Yes, I checked kernel parameters and they look ok:


    IPC Resource Limits, '/usr/bin/ipcs -l'


    ------ Shared Memory Limits --------
    max number of segments = 4096
    max seg size (kbytes) = 67108864
    max total shared memory (kbytes) = 17179869184
    min seg size (bytes) = 1

    ------ Semaphore Limits --------
    max number of arrays = 1024
    max semaphores per array = 250
    max semaphores system wide = 32000
    max ops per semop call = 32
    semaphore max value = 32767

    ------ Messages: Limits --------
    max queues system wide = 1024
    max size of message (bytes) = 65536
    default max size of queue (bytes) = 65536



    Instance "xxxxxxxxx" uses "64" bits and DB2 code release "SQL09055"
    with level identifier "06060107".
    Informational tokens are "DB2 v9.5.0.5", "s091123", "MI00316", Fix Pack "5".
    DATA #2 : System Info, 440 bytes
    System: Linux x88cpdc 6 2 x86_64
    CPU: total:4 online:4 Cores per socket:4 Threading degree per core:1
    Physical Memory(MB): total:16049 free:6648
    Virtual Memory(MB): total:17046 free:7645
    Swap Memory(MB): total:997 free:997
    Kernel Params: msgMaxMessageSize:65536 msgMsgMap:65536 msgMaxQueueIDs:1024
    msgNumberOfHeaders:65536 msgMaxQueueSize:65536
    msgMaxSegmentSize:16 shmMax:68719476736 shmMin:1 shmIDs:4096
    shmSegments:4096 semMap:32000 semIDs:1024 semNum:32000
    semUndo:32000 semNumPerID:250 semOps:32 semUndoSize:20
    semMaxVal:32767 semAdjustOnExit:32767
    Cur cpu time limit (seconds) = 0xFFFFFFFF
    Cur file size limit (bytes) = 0xFFFFFFFF
    Cur data size (bytes) = 0xFFFFFFFF
    Cur stack size (bytes) = 0x00A00000
    Cur core size (bytes) = 0x00000000
    Cur memory size (bytes) = 0xFFFFFFFF
    nofiles (descriptors) = 0x00000400



    STTM should be off, I don't see any activity in the STMM logs.


    I also made sure that their Linux env meets system requirements as per:
    IBM developerWorks: Wikis - Information Management - Red Hat Enterprise Linux (RHEL) 5 - DB2 9.5


    This problem occurs during db migration from v8 to v9.5 (via restore), during system catalog migration step - migrating existing tables or creating new ones. The db is very small (backup image is ~160MB if compressed during backup and ~1GB if uncompressed) so it's not clear why db2 is getting an out of memory from the OS... Restoring the same backup image back to v8 completed successfully (cat migration is not needed).

    They use VCS and some vormetric FS guard software (for encryption), don't know if this is related or not.


    Any idea on how to check for amount of pinned memory at the OS level?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •