Results 1 to 5 of 5
  1. #1
    Join Date
    Apr 2010
    Posts
    3

    Unanswered: Help please, Database went down

    Hi, am very new to DB2 9.7.2 as a database and would like to seek advice. I have a database that went down when we were loading data into it. Db2diag.log shows this:

    I don't quite get it why it says disk full when a df -h shows that none of my mounted filesystems are near 100%.

    Database is running in circular mode;

    Code:
    2012-09-18-20.49.44.656783+480 I53846614E828       LEVEL: Severe
    PID     : 17221                TID  : 140124536956672PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000          DB   : FLQ
    APPHDL  : 0-1678               APPID: 10.10.24.22.59841.120913050226
    AUTHID  : SAPFLQ  
    EDUID   : 50                   EDUNAME: db2agent (FLQ) 0
    FUNCTION: DB2 UDB, relation data serv, sqlrr_array_input, probe:210
    DATA #1 : SQLCA, PD_DB2_TYPE_SQLCA, 136 bytes
     sqlcaid : SQLCA     sqlcabc: 136   sqlcode: -952   sqlerrml: 0
     sqlerrmc: 
     sqlerrp : SQLRI039
     sqlerrd : (1) 0x80040003      (2) 0x00000003      (3) 0x00000000
               (4) 0x00000000      (5) 0xFFFFFB85      (6) 0x00000000
     sqlwarn : (1)      (2)      (3)      (4)        (5)       (6)    
               (7)      (8)      (9)      (10)        (11)     
     sqlstate:      
    
    
    2012-09-18-21.02.59.496295+480 E53849101E1162      LEVEL: Error (OS)
    PID     : 17221                TID  : 140113199752960PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000
    EDUID   : 173                  EDUNAME: db2loggw (FLQ) 0
    FUNCTION: DB2 UDB, oper system services, sqloseekwrite64, probe:40
    MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
              DIA8312C Disk was full.
    CALLED  : OS, -, pwrite                           OSERR: ENOSPC (28)
    DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
      File Handle              = 9
      File System Block Size   = 32768 bytes
      File System Type         = nfs
      File Handle Flags :
        Require Sector Align   = No
        DIO/CIO Mode           = Yes
        Raw Block Device       = No
        Reserved Handle        = No
        Flush On Close         = No
        Thread-Level Lock      = No
        Write-through Mode     = Yes
        File Not Tracked       = No
    DATA #2 : unsigned integer, 8 bytes
    4096
    DATA #3 : signed integer, 8 bytes
    111353856
    DATA #4 : signed integer, 8 bytes
    -1
    DATA #5 : String, 105 bytes
    Search for ossError*Analysis probe point after this log entry for further
    self-diagnosis of this problem.
    
    2012-09-18-21.02.59.499844+480 E53850264E1082      LEVEL: Error (OS)
    PID     : 17221                TID  : 140124637619968PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000          DB   : FLQ
    EDUID   : 151                  EDUNAME: db2loggr (FLQ) 0
    FUNCTION: DB2 UDB, oper system services, sqlobufreset, probe:10
    MESSAGE : ZRC=0x860F0003=-2045837309=SQLO_DERR "disk error occurred (DOS)"
              DIA8402C A disk error has occurred.
    CALLED  : OS, -, fsync                            OSERR: EIO (5)
    DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
      File Handle              = 496
      File System Block Size   = 0 bytes
      File System Type         = UNKNOWN
      File Handle Flags :
        Require Sector Align   = No
        DIO/CIO Mode           = No
        Raw Block Device       = No
        Reserved Handle        = No
        Flush On Close         = Yes
        Thread-Level Lock      = Yes
        Write-through Mode     = No
        File Not Tracked       = No
    DATA #2 : String, 105 bytes
    Search for ossError*Analysis probe point after this log entry for further
    self-diagnosis of this problem.
    
    2012-09-18-21.02.59.501823+480 I53851347E438       LEVEL: Error
    PID     : 17221                TID  : 140124637619968PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000          DB   : FLQ
    EDUID   : 151                  EDUNAME: db2loggr (FLQ) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgifl, probe:90
    RETCODE : ZRC=0x860F0003=-2045837309=SQLO_DERR "disk error occurred (DOS)"
              DIA8402C A disk error has occurred.
    
    2012-09-18-21.02.59.503927+480 E53851786E829       LEVEL: Severe (OS)
    PID     : 17221                TID  : 140124637619968PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000          DB   : FLQ
    EDUID   : 151                  EDUNAME: db2loggr (FLQ) 0
    FUNCTION: DB2 UDB, oper system services, sqloclose, probe:10
    CALLED  : OS, -, fsync                            OSERR: ENOSPC (28)
    DATA #1 : File handle, PD_TYPE_SQO_FILE_HDL, 8 bytes
      File Handle              = 496
      File System Block Size   = 0 bytes
      File System Type         = UNKNOWN
      File Handle Flags :
        Require Sector Align   = No
        DIO/CIO Mode           = No
        Raw Block Device       = No
        Reserved Handle        = No
        Flush On Close         = Yes
        Thread-Level Lock      = Yes
        Write-through Mode     = No
        File Not Tracked       = No
    
    2012-09-18-21.02.59.509508+480 I53852616E2567      LEVEL: Error (OS)
    PID     : 17221                TID  : 140113199752960PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000
    EDUID   : 173                  EDUNAME: db2loggw (FLQ) 0
    FUNCTION: DB2 Common, OSSe, ossErrorIOAnalysis, probe:100
    CALLED  : OS, -, pwrite                           OSERR: ENOSPC (28)
    DATA #1 : String, 145 bytes
    A total of 5 analysis will be performed :
     - User info
     - ulimit info
     - Target file info
     - I/O attempt
     - File system
    
     Target file handle = 9
    DATA #2 : String, 188 bytes
      Real user ID of current process       = 3604
      Effective user ID of current process  = 3604
      Real group ID of current process      = 3604
      Effective group ID of current process = 3604
    DATA #3 : String, 369 bytes
    Current process limits (unit in bytes except for nofiles) :
      mem     (S/H) = unlimited / unlimited
      core    (S/H) = unlimited / unlimited
      cpu     (S/H) = unlimited / unlimited
      data    (S/H) = unlimited / unlimited
      fsize   (S/H) = unlimited / unlimited
      nofiles (S/H) = 65536 / 65536
      stack   (S/H) = 10485760 / 10485760
      rss     (S/H) = unlimited / unlimited
    DATA #4 : String, 264 bytes
    Target File Information :
      Size               = 536748032
      Link               = No
      Reference path     = N/A
      Type               = 0x8000
      Permissions        = rw-------
      UID                = 3604
      GID                = 3604
      Last modified time = 1347973463
    DATA #5 : String, 33 bytes
    I/O attempt not implemented yet.
    DATA #6 : String, 403 bytes
    File System Information of the target file :
      Block size        = 32768 bytes
      Total size        = 25769803776 bytes
      Free size         = 0 bytes
      Total # of inodes = 933880
      FS name           = nas02-st:/vol/saplog_p1_FLQ/p1
      Mount point       = /saplog/log_dir/FLQ
      FSID              = 57
      FS type name      = nfs
      DIO/CIO mount opt = None
      Device type       = N/A
      FS type           = 0x1
    CALLSTCK: 
      [0] 0x00007F715B50DD96 pdOSSeLoggingCallback + 0x20C
      [1] 0x00007F715A190A92 /home_sap/db2flq/sqllib/lib64/libdb2osse.so.1 + 0x1C4A92
      [2] 0x00007F715A190989 ossLogSysRC + 0xBF
      [3] 0x00007F715A185AFE /home_sap/db2flq/sqllib/lib64/libdb2osse.so.1 + 0x1B9AFE
      [4] 0x00007F715A182E7D ossErrorAnalysis + 0x2D
      [5] 0x00007F715CEFBEC9 sqloSystemErrorHandler + 0x6D7
      [6] 0x00007F715BA351E2 sqloseekwrite64 + 0x66E
      [7] 0x00007F715BA348C6 sqloWriteBlocks + 0xCC
      [8] 0x00007F715BA7746F /home_sap/db2flq/sqllib/lib64/libdb2e.so.1 + 0x144246F
      [9] 0x00007F715BA77077 _Z13sqlpgPingPongP9SQLP_DBCBP9SQLP_LFPBmm + 0x121
    
    2012-09-18-21.02.59.588438+480 E53855184E473       LEVEL: Severe
    PID     : 17221                TID  : 140113199752960PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000
    EDUID   : 173                  EDUNAME: db2loggw (FLQ) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgWriteToDisk, probe:1370
    MESSAGE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
              DIA8312C Disk was full.
    DATA #1 : <preformatted>
    Error while writing to log file S0000046.LOG 
    
    2012-09-18-21.02.59.617948+480 I53855658E399       LEVEL: Severe
    PID     : 17221                TID  : 140113199752960PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000
    EDUID   : 173                  EDUNAME: db2loggw (FLQ) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgPingPong, probe:1270
    RETCODE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
              DIA8312C Disk was full.
    
    2012-09-18-21.02.59.618913+480 I53856058E394       LEVEL: Severe
    PID     : 17221                TID  : 140113199752960PROC : db2sysc 0
    INSTANCE: db2flq               NODE : 000
    EDUID   : 173                  EDUNAME: db2loggw (FLQ) 0
    FUNCTION: DB2 UDB, data protection services, sqlpgwlp, probe:1624
    RETCODE : ZRC=0x850F000C=-2062614516=SQLO_DISK "Disk full."
              DIA8312C Disk was full.

  2. #2
    Join Date
    Apr 2010
    Posts
    3
    Weird part is that S0000046.LOG doesn't even exist? Help please!

    db2 => ffvm002:db2flq 66> ls -ltr
    total 10524724
    -rw------- 1 db2flq dbflqadm 512 Sep 12 17:12 SQLLPATH.TAG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000001.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000002.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000003.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000004.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000005.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000006.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000007.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000008.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000009.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000010.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000011.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000012.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000013.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000014.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000015.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000016.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000017.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000018.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 10:11 S0000019.LOG
    -rw------- 1 db2flq dbflqadm 536748032 Sep 19 2012 S0000000.LOG
    ffvm002:db2flq 67> pwd
    /saplog/log_dir/FLQ/NODE0000
    ffvm002:db2flq 68>

  3. #3
    Join Date
    Apr 2006
    Location
    Belgium
    Posts
    2,514
    Provided Answers: 11
    loading to table that already contained data ?
    probably temp space usage for rebuild of index
    while doing load check usage of filesystem filling
    check also tablespace name in diaglog - if tempspace then container like ........xx.TMP
    are you using nfs as database filesystem
    File System Type = nfs
    for logging ?? maybe nbr/size of logs overcommitted
    Best Regards, Guy Przytula
    Database Software Consultant
    Good DBAs are not formed in a week or a month. They are created little by little, day by day. Protracted and patient effort is needed to develop good DBAs.
    Spoon feeding : To treat (another) in a way that discourages independent thought or action, as by overindulgence.
    DB2 UDB LUW Certified V7-V8-V9-V9.7-V10.1-V10.5 DB Admin - Advanced DBA -Dprop..
    Information Server Datastage Certified
    http://www.infocura.be

  4. #4
    Join Date
    Oct 2007
    Posts
    246
    loading table which already having data , and if ur loading into key column then it will delete
    Drop the index for the tables which ur loading and recreate after loading the data

    Diag error say Error while writing to Log : Disk Full pls chk the file system for log

    paul

  5. #5
    Join Date
    Jun 2003
    Location
    Toronto, Canada
    Posts
    5,516
    Provided Answers: 1
    Quote Originally Posted by cookies2648 View Post
    when we were loading data into it.
    Looks like there was a long-running transaction that generated enough logs to fill 25 GB in /saplog/log_dir/FLQ. Try committing more often, or use the LOAD utilitiy.
    ---
    "It does not work" is not a valid problem statement.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •