Results 1 to 7 of 7
  1. #1
    Join Date
    Aug 2011
    Location
    Columbus, OH
    Posts
    55

    Unanswered: DB2 disk full error using SMS

    Stats:
    "DB2 v8.1.1.128", "s061108", "U810098", and FixPak "14"
    "DB2 Enterprise Server Edition"
    AIX 5300-11-03-1013

    Problem:
    During normal processing, applications began failing as the db2diag.log reported errors that the disk is full. See below for the exact error messages.

    All tablespaces are SMS and there is 9.73 GB or 46% free space on the directory, so there should be no issue there.

    It's odd to note that the SQL00003.DAT file in tablespace directory is .9999999 GB (strange that it seems to fail at the 1GB threshold)


    Additional problem with similar issue:


    When performing a backup, the backup must be spread over 3 files. It seems that once a backup file becomes great than 1GB it must be broken up.

    $ db2 backup db cmdlilsp online to '/db2backup/CMDLILSP/daily' compress include logs without prompting
    SQL2419N The target disk "/db2backup/CMDLILSP/daily" has become full.

    but this comand works fine....
    db2 backup db cmdlilsp online to '/db2backup/CMDLILSP/daily','/db2backup/CMDLILSP/daily','/db2backup/CMDLILSP/daily' compress include logs without prompting;


    Things I've checked:
    I've check the db2set and env paramaters as well as the dbm and db cfg's, nothing that I've found would make me think there is a 1GB limitation.

    I've had the AIX admin check to make sure there is no limitation for the db2 instance owner or any other user.


    Table space growth errors:


    2011-08-18-08.44.45.010664-240 I26681339C760 LEVEL: Error
    PID : 36808 TID : 1 PROC : db2agent (CMDLILSP) 0
    INSTANCE: cmililsp NODE : 000 DB : CMDLILSP
    APPHDL : 0-1201 APPID: GA651871.IB06.103FC8105522
    FUNCTION: DB2 UDB, oper system services, sqloseekwrite64, probe:100
    MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
    DIA8312C Disk was full.
    DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
    0x2FF19108 : 0000 0009 0000 0000 ........
    DATA #2 : unsigned integer, 4 bytes
    32768
    DATA #3 : signed integer, 8 bytes
    1073709056
    DATA #4 : unsigned integer, 4 bytes
    0
    DATA #5 : Hex integer, 4 bytes
    0x00000000
    DATA #6 : signed integer, 4 bytes
    32256

    2011-08-18-08.44.45.205610-240 E26682100C777 LEVEL: Error
    PID : 36808 TID : 1 PROC : db2agent (CMDLILSP) 0
    INSTANCE: cmililsp NODE : 000 DB : CMDLILSP
    APPHDL : 0-1201 APPID: GA651871.IB06.103FC8105522
    FUNCTION: DB2 UDB, buffer pool services, sqlbWritePageToDisk, probe:20
    MESSAGE : ADM6017E The table space "ICMLNF32" (ID "4") is full. Detected on
    container "/cmdata/cmililsp/NODE0000/SQL00001/ICMLNF32" (ID "0").
    The underlying file system is full or the maximum allowed space usage
    for the file system has been reached. It is also possible that there
    are user limits in place with respect to maximum file size and these
    limits have been reached.e



    Backup errors:


    2011-08-22-15.46.16.005807-240 I34457679C385 LEVEL: Warning
    PID : 30446 TID : 1 PROC : db2agent (idle) 0
    INSTANCE: cmililsp NODE : 000 DB : CMDLILSP
    APPHDL : 0-1106 APPID: *LOCAL.cmililsp.110822194615
    FUNCTION: DB2 UDB, database utilities, sqlubSetupJobControl, probe:2028
    MESSAGE : Starting an online db backup.

    2011-08-22-15.48.16.113401-240 I34458065C573 LEVEL: Error
    PID : 41002 TID : 1 PROC : db2med.30446.0 0
    INSTANCE: cmililsp NODE : 000
    FUNCTION: DB2 UDB, oper system services, sqlowrite, probe:200
    MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
    DIA8312C Disk was full.
    DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
    0x2FF21098 : 0000 0003 0000 0000 ........
    DATA #2 : unsigned integer, 4 bytes
    1044480
    DATA #3 : signed integer, 4 bytes
    11776
    DATA #4 : signed integer, 4 bytes
    0

    2011-08-22-15.48.16.157595-240 I34458639C330 LEVEL: Warning
    PID : 41002 TID : 1 PROC : db2med.30446.0 0
    INSTANCE: cmililsp NODE : 000
    FUNCTION: DB2 UDB, database utilities, sqluMCWriteToDevice, probe:60
    MESSAGE : Media controller -- Disk full encountered on /db2backup/CMDLILSP/dail

  2. #2
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    Check user limits for your instance owner id ( ulimit -a ). Look for:

    file(blocks)

  3. #3
    Join Date
    Aug 2011
    Location
    Columbus, OH
    Posts
    55

    ulimit for db2 instance owner

    $ whoami
    cmililsp
    $ ulimit -a
    time(seconds) unlimited
    file(blocks) unlimited
    data(kbytes) 245760
    stack(kbytes) 16384
    memory(kbytes) unlimited
    coredump(blocks) unlimited
    nofiles(descriptors) 2000


    I'm not an AIX wizard, so I don't know if any of these look too low.

  4. #4
    Join Date
    Jun 2003
    Location
    Toronto, Canada
    Posts
    5,516
    Provided Answers: 1
    As the user cmililsp, try to create a file larger than 1 GB in size in one of the directories where DB2 encounters an error. I would expect it to fail; if that's the case then there IS a limit imposed by the operating system.

    Try also running a "quota" command (without any parameters) - it will show if there's a disk quota set for the login ID.

  5. #5
    Join Date
    Aug 2011
    Location
    Columbus, OH
    Posts
    55

    Thanks for the help so far!

    I successfully copied a > 1 GB file from another server to the backup directory while signed on as the db2 intance owner.

    $ quota
    Disk quotas for user cmililsp (uid 207): none


    I'm thinking the OS is innocent, could DB2 be responsible in any way for the limitaiton? I'm a still a new DBA, so I'm not sure I've checked everywhere I should have.

    With this version of DB2 being out of support, I'm not even able to open a pmr with IBM for assistance. Any other thoughts? I really appreciate the help so far. If nothing else, I've learned some new unix commands

  6. #6
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    Check my last update here:
    http://www.dbforums.com/db2/1645716-...rge-table.html


    Use the information in the technote to check the ulimits in effect (pstat command). I suspect it will show 1GB. Or if you can, restart the instance using your instance owner id and then retry the backup command (with one target dir).

  7. #7
    Join Date
    Aug 2011
    Location
    Columbus, OH
    Posts
    55

    db2girl rocks!!!

    db2girl, you rock! Thank you so much for pointing me to this post. It was the source of my problem and now I have a fix. THANK YOU!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •