Unanswered: table got stuck in RO mode and we dont know how!
We dropped/recreated a table, ran runstats, IC's and thought all was fine. The next morning we got a call that apps jobs were failing. I did a display and found the table in Read Only mode.
We drop/recreate tables all the time, we have never had this happen before. Researched the IC job, runstats job, all completed successfully.
We have mainview and apptune in our shop, nothing in there to indicate anything ran to put the table in RO. We checked all the jobs that the apps team runs from the time of our drop/recreate until I got called, still nothing. We also checked the DB logs, still no evidence that it was put in RO.
We even tried to recreate the problem in test. We noticed when you run runstats, the table is thrown in RO for a few seconds while it runs, but when it completes, regardless of whether it is successful or not, the tablespace was put back to RW.
We are totally stumped. Anyone ever have this happen? Any help or suggestions would be greatly appreciated.
I hope no one is looking into this but I thought I would post our findings.
We replicated EXACTLY what we did in prod in test and we found that after BMC change manager is the culprit. After we loaded the data, we ran runstats and then the IC's which blew up on a space issue. Sure enough I displayed the tablespace and it was in RO,COPY
What we did wrong was instead of fixing the space issue and restarting the BMC job, we just manually ran IC's for each partition, which took it out of COPY but left it in RO.
Actually what we really did wrong was not displaying the tablespace after all was done, we would have seen the RO status and could have flipped it right then and there.