Our RAC environment automatically rebooted one of our nodes, thank goodness it happened while there are no users connected. We assumed it is caused by node evictions, and suspected disk ASM disk unavailability, however we cannot find a concrete cause why this happened. and what can we do to prevent such incidents? Provided logs below for reference.

from alert_node2.log

2014-09-08 22:46:41.300:
[client(2687626)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2014-09-08 22:46:41.302:
[client(2687626)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/apps/orclrac/grid/11203/log/uphod01/client/crsctl_orclrac.log.
2014-09-08 22:46:45.046:
[client(721704)]CRS-2302:Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2014-09-08 22:46:45.048:
[client(721704)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/apps/orclrac/grid/11203/log/uphod01/client/crsctl_orclrac.log.
2014-09-08 22:47:30.185:
[ohasd(983946)]CRS-2112:The OLR service started on node uphod01.
2014-09-08 22:47:30.336:
[ohasd(983946)]CRS-1301:Oracle High Availability Service started on node uphod01.
2014-09-08 22:47:30.343:
[ohasd(983946)]CRS-8011:reboot advisory message from host: uphod01, component: cssmonit, with time stamp: L-2014-09-08-22:37:48.103
[ohasd(983946)]CRS-8013:reboot advisory message text: Rebooting after limit 28239 exceeded; disk timeout 28239, network timeout 26670, last heartbeat from CSSD at epoch seconds 1410187039.824, 28279 milliseconds ago based on invariant clock value of 2370576665
2014-09-08 22:47:30.343:
[ohasd(983946)]CRS-8017:location: /etc/oracle/lastgasp has 2 reboot advisory log files, 1 were announced and 0 errors occurred
2014-09-08 22:47:43.059:
[/u01/apps/orclrac/grid/11203/bin/oraagent.bin(3211802)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006" in "/u01/apps/orclrac/grid/11203/log/uphod01/agent/ohasd/oraagent_orclrac/oraagent_orclrac.log"

from crsctl_<user>.log

[ CLWAL][1]clsw_Initialize: Error [PROCL-32: Oracle High Availability Services on the local node is not running Messaging error [gipcretConnectionRefused] [29]] from procr_init_ext
2014-09-08 22:46:36.319: [ GPNP][1]clsgpnpkww_initclswcx: [at clsgpnpkww.c:351 clsgpnpkww_initclswcx] Result: (56) CLSGPNP_OCR_INIT. Failed to init CLSW-OLR context. CLSW Error (3): CLSW-3: Error in the cluster registry (OCR) layer. [32] [PROCL-32: Oracle High Availability Services on the local node is not running Messaging error [gipcretConnectionRefused] [29]]
2014-09-08 22:46:36.355: [ default][1]Cannot get GPnP profile. Error CLSGPNP_NO_DAEMON (GPNPD daemon is not running).
2014-09-08 22:46:36.356: [ OCRASM][1]proprasmo: Error [13] in opening the GPNP profile. Try to get offline profile
2014-09-08 22:46:36.356: [ GPNP][1]clsgpnp_getOfflineProfile: [at clsgpnp.c:584 clsgpnp_getOfflineProfile] Result: (8) CLSGPNP_PERMS. Must be a privileged user to get an offline GPnP profile.
2014-09-08 22:46:36.356: [ OCRASM][1]proprasmo: Error [8] in opening the GPNP offline profile.
2014-09-08 22:46:36.356: [ OCRASM][1]proprasmo: Error in open/create file in dg [DATA_OCR]
[ OCRASM][1]SLOS : SLOS: cat=7, opn=kgfoAl06, dep=29701, loc=kgfokge
2014-09-08 22:46:36.356: [ OCRASM][1]ASM Error Stack : ORA-29701: unable to connect to Cluster Synchronization Service
2014-09-08 22:46:36.358: [ OCRASM][1]proprasmo: kgfoCheckMount returned [7]
2014-09-08 22:46:36.358: [ OCRASM][1]proprasmo: The ASM instance is down
2014-09-08 22:46:36.558: [ OCRRAW][1]proprioo: Failed to open [+DATA_OCR]. Returned proprasmo() with [26]. Marking location as UNAVAILABLE.
2014-09-08 22:46:36.558: [ OCRRAW][1]proprioo: No OCR/OLR devices are usable
2014-09-08 22:46:36.558: [ OCRASM][1]proprasmcl: asmhandle is NULL
2014-09-08 22:46:36.558: [ OCRRAW][1]proprinit: Could not open raw device
2014-09-08 22:46:36.558: [ OCRASM][1]proprasmcl: asmhandle is NULL
2014-09-08 22:46:36.558: [ default][1]a_init:7!: Backend init unsuccessful : [26]
2014-09-08 22:46:38.212: [ OCRMSG][1]prom_waitconnect: CONN NOT ESTABLISHED (0,29,1,2)
2014-09-08 22:46:38.212: [ OCRMSG][1]GIPC error [29] msg [gipcretConnectionRefused]
2014-09-08 22:46:38.212: [ OCRMSG][1]prom_connect: error while waiting for connection complete [24]
[ CLWAL][1]clsw_Initialize: OLR initlevel [30000]
2014-09-08 22:46:41.264: [ OCRMSG][1]prom_waitconnect: CONN NOT ESTABLISHED (0,29,1,2)
2014-09-08 22:46:41.264: [ OCRMSG][1]GIPC error [29] msg [gipcretConnectionRefused]
2014-09-08 22:46:41.264: [ OCRMSG][1]prom_connect: error while waiting for connection complete [24]
[ CLWAL][1]clsw_Initialize: Error [32] from procr_init_ext
[ CLWAL][1]clsw_Initialize: Error [PROCL-32: Oracle High Availability Services on the local node is not running Messaging error [gipcretConnectionRefused] [29]] from procr_init_ext
2014-09-08 22:46:41.264: [ GPNP][1]clsgpnpkww_initclswcx: [at clsgpnpkww.c:351 clsgpnpkww_initclswcx] Result: (56) CLSGPNP_OCR_INIT. Failed to init CLSW-OLR context. CLSW


Will appreciate your answers! thank you!