Oracle Restart Upgrade Failed simply because CRS diskgroup is OFFLINE.
ora.CRS.dg
OFFLINE OFFLINE db-asm-1 STABLE
Resolution coming up.
Upgrade Log
[oracle@db-asm-1 ~]$ crsctl query has releaseversion
Oracle High Availability Services release version on the local node is [11.2.0.4.0]
[oracle@db-asm-1 ~]$ crsctl query has softwareversion
Oracle High Availability Services version on the local node is [11.2.0.4.0]
[oracle@db-asm-1 ~]$
00:58:42 SYS @ +ASM:>show parameter pfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +CRS/asm/asmparameterfile/regi
stry.253.956854045
00:58:47 SYS @ +ASM:>show parameter disk
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string DATA, FRA
asm_diskstring string /dev/oracle/*
00:58:52 SYS @ +ASM:>
[oracle@db-asm-1 ~]$ srvctl config asm
ASM home: /u01/app/oracle/11.2.0.4/grid
ASM listener: LISTENER
Spfile: +CRS/asm/asmparameterfile/registry.253.956854045
ASM diskgroup discovery string: /dev/oracle/*
[oracle@db-asm-1 ~]$ asmcmd lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 4194304 10236 10148 0 10148 0 N CRS/
MOUNTED EXTERN N 512 4096 4194304 20472 18612 0 18612 0 N DATA/
MOUNTED EXTERN N 512 4096 4194304 30708 30264 0 30264 0 N FRA/
[oracle@db-asm-1 ~]$
[oracle@db-asm-1 ~]$ env|egrep -i 'oracle|home'
USER=oracle
LD_LIBRARY_PATH=/u01/app/oracle/11.2.0.4/grid/lib
ORACLE_SID=+ASM
ORACLE_BASE=/u01/app/oracle
MAIL=/var/spool/mail/oracle
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/oracle/bin:/u01/app/oracle/11.2.0.4/grid/bin
PWD=/home/oracle
HOME=/home/oracle
GRID_HOME=/u01/app/oracle/11.2.0.4/grid
LOGNAME=oracle
LOG=/u01/app/oracle/cfgtoollogs/catbundle/catbundle_PSU_ORCLCDB_APPLY_2017Oct08_21_41_02.log
ORACLE_HOME=/u01/app/oracle/11.2.0.4/grid
[oracle@db-asm-1 ~]$ unset ORACLE_SID ORACLE_BASE GRID_HOME ORACLE_HOME
[oracle@db-asm-1 ~]$ env|egrep -i 'oracle|home'
USER=oracle
LD_LIBRARY_PATH=/u01/app/oracle/11.2.0.4/grid/lib
MAIL=/var/spool/mail/oracle
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/oracle/bin:/u01/app/oracle/11.2.0.4/grid/bin
PWD=/home/oracle
HOME=/home/oracle
LOGNAME=oracle
LOG=/u01/app/oracle/cfgtoollogs/catbundle/catbundle_PSU_ORCLCDB_APPLY_2017Oct08_21_41_02.log
[oracle@db-asm-1 ~]$
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[oracle@db-asm-1 ~]$ cd /u01/app/12.2.0.1/grid/
[oracle@db-asm-1 grid]$ ./gridSetup.sh -silent -waitforcompletion -skipPrereqs -responseFile /u01/app/12.2.0.1/grid/install/response/grid_upgrade.rsp
Launching Oracle Grid Infrastructure Setup Wizard...
[WARNING] [INS-40436] Installer has detected an earlier version of Oracle Automatic Storage Management (ASM) Instance configured. Before proceeding with the upgrade you must shutdown all Oracle Databases that are using Oracle ASM for storage.
You may ignore this message and continue with the upgrade, if the database(s) are already in shutdown state.
[WARNING] [INS-41808] Possible invalid choice for OSASM Group.
CAUSE: The name of the group you selected for the OSASM group is commonly used to grant other system privileges (For example: asmdba, asmoper, dba, oper).
ACTION: Oracle recommends that you designate asmadmin as the OSASM group.
[WARNING] [INS-41809] Possible invalid choice for OSDBA Group.
CAUSE: The group name you selected as the OSDBA for ASM group is commonly used for Oracle Database administrator privileges.
ACTION: Oracle recommends that you designate asmdba as the OSDBA for ASM group, and that the group should not be the same group as an Oracle Database OSDBA group.
[WARNING] [INS-41810] Possible invalid choice for OSOPER Group.
CAUSE: The group name you selected as the OSOPER for ASM group is commonly used for Oracle Database administrator privileges.
ACTION: Oracle recommends that you designate asmoper as the OSOPER for ASM group, and that the group should not be the same group as an Oracle Database OSOPER group.
[WARNING] [INS-41812] OSDBA and OSASM are the same OS group.
CAUSE: The chosen values for OSDBA group and the chosen value for OSASM group are the same.
ACTION: Select an OS group that is unique for ASM administrators. The OSASM group should not be the same as the OS groups that grant privileges for Oracle ASM access, or for database administration.
[WARNING] [INS-32018] The selected Oracle home is outside of Oracle base.
ACTION: Oracle recommends installing Oracle software within the Oracle base directory. Adjust the Oracle home or Oracle base accordingly.
You can find the log of this install session at:
/u01/app/oraInventory/logs/GridSetupActions2017-10-09_01-03-02AM/gridSetupActions2017-10-09_01-03-02AM.log
As a root user, execute the following script(s):
1. /u01/app/12.2.0.1/grid/rootupgrade.sh
Execute /u01/app/12.2.0.1/grid/rootupgrade.sh on the following nodes:
[db-asm-1]
Successfully Setup Software.
As install user, execute the following command to complete the configuration.
/u01/app/12.2.0.1/grid/gridSetup.sh -executeConfigTools -responseFile /u01/app/12.2.0.1/grid/install/response/grid_upgrade.rsp [-silent]
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[oracle@db-asm-1 grid]$ ps -ef|grep pmon
oracle 3333 26998 0 01:19 pts/2 00:00:00 grep pmon
oracle 14084 1 0 Oct08 ? 00:00:02 asm_pmon_+ASM
oracle 14346 1 0 Oct08 ? 00:00:02 ora_pmon_orclcdb
[oracle@db-asm-1 grid]$ asmcmd lsdg
Connected to an idle instance.
ASMCMD-8102: no connection to Oracle ASM; command requires Oracle ASM to run
[oracle@db-asm-1 grid]$ srvctl status asm
ASM is running on db-asm-1
[oracle@db-asm-1 grid]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
ONLINE ONLINE db-asm-1
ora.DATA.dg
ONLINE ONLINE db-asm-1
ora.FRA.dg
ONLINE ONLINE db-asm-1
ora.LISTENER.lsnr
ONLINE ONLINE db-asm-1
ora.asm
ONLINE ONLINE db-asm-1 Started
ora.ons
OFFLINE OFFLINE db-asm-1
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE db-asm-1
ora.diskmon
1 OFFLINE OFFLINE
ora.evmd
1 ONLINE ONLINE db-asm-1
ora.orclcdb.db
1 ONLINE ONLINE db-asm-1 Open
[oracle@db-asm-1 grid]$ env|egrep -i 'oracle|home'
OLDPWD=/home/oracle
USER=oracle
LD_LIBRARY_PATH=/u01/app/oracle/11.2.0.4/grid/lib
MAIL=/var/spool/mail/oracle
PATH=/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/oracle/bin:/u01/app/oracle/11.2.0.4/grid/bin
HOME=/home/oracle
LOGNAME=oracle
LOG=/u01/app/oracle/cfgtoollogs/catbundle/catbundle_PSU_ORCLCDB_APPLY_2017Oct08_21_41_02.log
[oracle@db-asm-1 grid]$
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[oracle@db-asm-1 ~]$ su -
Password:
[root@db-asm-1 ~]# /u01/app/12.2.0.1/grid/rootupgrade.sh
Check /u01/app/12.2.0.1/grid/install/root_db-asm-1_2017-10-09_01-22-49-043808622.log for the output of root script
[root@db-asm-1 ~]# ps -ef|grep pmon
root 8648 3386 0 01:32 pts/1 00:00:00 grep pmon
[root@db-asm-1 ~]#
[oracle@db-asm-1 ~]$ cat /u01/app/12.2.0.1/grid/install/root_db-asm-1_2017-10-09_01-22-49-043808622.log
Performing root user operation.
The following environment variables are set as:
ORACLE_OWNER= oracle
ORACLE_HOME= /u01/app/12.2.0.1/grid
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/12.2.0.1/grid/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/oracle/crsdata/db-asm-1/crsconfig/roothas_2017-10-09_01-22-50AM.log
2017/10/09 01:23:00 CLSRSC-363: User ignored prerequisites during installation
ASM has been upgraded and started successfully.
Creating OCR keys for user 'oracle', privgrp 'oinstall'..
Operation successful.
LOCAL ONLY MODE
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4664: Node db-asm-1 successfully pinned.
2017/10/09 01:27:19 CLSRSC-329: Replacing Clusterware entries in file 'oracle-ohasd.conf'
CRS-4123: Oracle High Availability Services has been started.
2017/10/09 01:29:31 CLSRSC-482: Running command: 'srvctl upgrade model -s 11.2.0.4.0 -d 12.2.0.1.0 -p first'
2017/10/09 01:29:52 CLSRSC-482: Running command: 'srvctl upgrade model -s 11.2.0.4.0 -d 12.2.0.1.0 -p last'
db-asm-1 2017/10/09 01:29:57 /u01/app/12.2.0.1/grid/cdata/db-asm-1/backup_20171009_012957.olr 0
db-asm-1 2017/10/08 16:45:27 /u01/app/oracle/11.2.0.4/grid/cdata/db-asm-1/backup_20171008_164527.olr -
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'db-asm-1'
CRS-2673: Attempting to stop 'ora.evmd' on 'db-asm-1'
CRS-2677: Stop of 'ora.evmd' on 'db-asm-1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'db-asm-1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/10/09 01:31:14 CLSRSC-327: Successfully configured Oracle Restart for a standalone server
[oracle@db-asm-1 ~]$
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[oracle@db-asm-1 grid]$ ps -ef|grep pmon
oracle 8652 26998 0 01:33 pts/2 00:00:00 grep pmon
[oracle@db-asm-1 grid]$ /u01/app/12.2.0.1/grid/gridSetup.sh -executeConfigTools -responseFile /u01/app/12.2.0.1/grid/install/response/grid_upgrade.rsp -silent
Launching Oracle Grid Infrastructure Setup Wizard...
You can find the logs of this session at:
/u01/app/oraInventory/logs/GridSetupActions2017-10-09_01-33-40AM
Successfully Configured Software.
[oracle@db-asm-1 grid]$ cd /u01/app/oraInventory/logs/GridSetupActions2017-10-09_01-33-40AM
[oracle@db-asm-1 GridSetupActions2017-10-09_01-33-40AM]$ ls -lrt
total 80
-rw-r-----. 1 oracle oinstall 129 Oct 9 01:33 installerPatchActions_2017-10-09_01-33-40AM.log
-rw-r-----. 1 oracle oinstall 0 Oct 9 01:33 gridSetupActions2017-10-09_01-33-40AM.err
-rw-r-----. 1 oracle oinstall 196 Oct 9 01:35 gridSetupActions2017-10-09_01-33-40AM.out
-rw-r-----. 1 oracle oinstall 2176 Oct 9 01:35 time2017-10-09_01-33-40AM.log
-rw-r-----. 1 oracle oinstall 67881 Oct 9 01:35 gridSetupActions2017-10-09_01-33-40AM.log
[oracle@db-asm-1 GridSetupActions2017-10-09_01-33-40AM]$ ps -ef|grep pmon
oracle 9335 26998 0 01:37 pts/2 00:00:00 grep pmon
[oracle@db-asm-1 GridSetupActions2017-10-09_01-33-40AM]$ tail /etc/oratab
# The first and second fields are the system identifier and home
# directory of the database respectively. The third filed indicates
# to the dbstart utility that the database should , "Y", or should not,
# "N", be brought up at system boot time.
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
orclcdb:/u01/app/oracle/11.2.0.4/db1:N # line added by Agent
+ASM:/u01/app/12.2.0.1/grid:N # line added by Agent
[oracle@db-asm-1 GridSetupActions2017-10-
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[oracle@db-asm-1 ~]$ . oraenv <<< +ASM
ORACLE_SID = [oracle] ? The Oracle base has been set to /u01/app/oracle
[oracle@db-asm-1 ~]$ echo $ORACLE_HOME
/u01/app/12.2.0.1/grid
[oracle@db-asm-1 ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
OFFLINE OFFLINE db-asm-1 STABLE
ora.DATA.dg
ONLINE OFFLINE db-asm-1 STABLE
ora.FRA.dg
ONLINE OFFLINE db-asm-1 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE db-asm-1 STABLE
ora.asm
ONLINE OFFLINE db-asm-1 STABLE
ora.ons
OFFLINE OFFLINE db-asm-1 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
1 ONLINE ONLINE db-asm-1 STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.evmd
1 ONLINE ONLINE db-asm-1 STABLE
ora.orclcdb.db
1 ONLINE OFFLINE Instance Shutdown,ST
ABLE
--------------------------------------------------------------------------------
[oracle@db-asm-1 ~]$ ps -ef|grep pmon
oracle 9360 4055 0 01:39 pts/3 00:00:00 grep pmon
[oracle@db-asm-1 ~]$ asmcmd lsdg
Connected to an idle instance.
ASMCMD-8102: no connection to Oracle ASM; command requires Oracle ASM to run
[oracle@db-asm-1 ~]$ crsctl query has releaseversion
Oracle High Availability Services release version on the local node is [12.2.0.1.0]
[oracle@db-asm-1 ~]$ crsctl query has softwareversion
Oracle High Availability Services version on the local node is [12.2.0.1.0]
[oracle@db-asm-1 ~]$
