Results 1 to 5 of 5
  1. #1
    Join Date
    Jun 2009
    Posts
    272

    Unanswered: Online database backup failing

    Environment:

    DPF with 17 database partitions spanned across 3 physical nodes. 1 partition on Admin node, 8 partitions each on two data nodes.

    Operating system: Linux 2.6.18-164.2.1.el5

    Issue:

    When trying to run an online database backup, the backup is failing. The backup is actually getting interrupted as it got failed on one of the partition with the below error. Highlighted in red.

    Looks like it is something related to maxfilop parameter. Can some one please suggest how to adjust this parameter and how to verify how many files are actually being open and what is the operating system limit for that ??

    0000 SQL2001N The utility was interrupted. The output data may be incomplete.
    0001 SQL2001N The utility was interrupted. The output data may be incomplete.
    0002 SQL2001N The utility was interrupted. The output data may be incomplete.
    0003 SQL2001N The utility was interrupted. The output data may be incomplete.
    0004 SQL2001N The utility was interrupted. The output data may be incomplete.
    0005 SQL2001N The utility was interrupted. The output data may be incomplete.
    0006 SQL0931C Operating system file table overflow occurred. Subsequent SQL statements cannot be processed.0007 SQL2001N The utility was interrupted. The output data may be incomplete.
    0008 SQL2001N The utility was interrupted. The output data may be incomplete.
    0009 SQL2001N The utility was interrupted. The output data may be incomplete.
    0010 SQL2001N The utility was interrupted. The output data may be incomplete.
    0011 SQL2001N The utility was interrupted. The output data may be incomplete.
    0012 SQL2001N The utility was interrupted. The output data may be incomplete.
    0013 SQL2001N The utility was interrupted. The output data may be incomplete.
    0014 SQL2001N The utility was interrupted. The output data may be incomplete.
    0015 SQL2001N The utility was interrupted. The output data may be incomplete.
    0016 SQL2001N The utility was interrupted. The output data may be incomplete.

    SQL2429N The database backup failed. The following database partitions
    returned errors: "6".

  2. #2
    Join Date
    Aug 2001
    Location
    UK
    Posts
    4,650
    Hope this explains

    SQL0931C Operating system file table overflow occurred. Subsequent SQL statements cannot be processed.

    Explanation: An Operating system limit has been reached. The application program is not permitted to issue additional SQL statements. The database is marked as needing recovery and all applications using the database are prevented from accessing the database.

    User Response: Terminate all the applications using the database. Restart the database.

    To help prevent the problem from recurring:

    Change the MAXFILOP database configuration parameter to a smaller value (this will reduce DB2's use of the operating system file table), and/or
    Terminate other applications that are using files, if appropriate, and/or,
    Refer to the operating system documentation for increasing the operating system file table limit. In most UNIX environments this can be accomplished by updating the kernel configuration with a larger value. (On AIX, this may only be possible by increasing the amount of memory in your machine).
    sqlcode: -931
    Visit the new-look IDUG Website , register to gain access to the excellent content.

  3. #3
    Join Date
    Jun 2009
    Posts
    272
    Well, that is not I was looking for. I would get that if I see what that error message is. Byt my question is how do we determine what is the linux operating system limit for file descriptors and currently how many file descriptors are being used ? If I get to know how to look at them. I can monitor the number of file descriptors being used while running my online backup and make sure they are not crossing the operating system limit.

  4. #4
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    On Linux, check file-max / ulimit -n. To check the number of file desrcriptors being used, I use lsof on AIX (example: lsof -p <PID of db2sysc> | wc -l). I believe the same command works on Linux.

  5. #5
    Join Date
    Jun 2009
    Posts
    272
    Yes, Bella you are right. I checked the file-max in /etc/sysctl.conf. And I was also issuing cat /proc/sys/fs/file-nr to determine the usage. At the same time I also monitored lsof | wc -l and found that it was crossing the threshold . This issue started happening becuase we migrated our database to 9.7 from 9.1. As we now 9.7 is a thread based and maxfilop would be changed from 64 to 61440 by default. Now we are planning to increase file-max to avoid this issue.

    Thanks for your help Bella.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •