Achive Stuck Situation and Troubleshooting

Embed Size (px)

Citation preview

  • 8/2/2019 Achive Stuck Situation and Troubleshooting

    1/5

    Related Sap Note No. 391

    Archive Stuck Situation and Troubleshooting

    Archive Stuck Situation Symptom: When in SAP R/3, archive stuck situation arise, R3

    stops responding, no user will be able to login to the system, update process will be hang up,all users who are logged in system will also be hang.

    Physical Symptom: At os level /oracle//oraarch or /oracle//saparch directory will

    be full of archive log files. There will not be free space available in above volumes.

    Situation when archive stuck can arise: When there is big client copy is under progress or

    big background batch input job is running or any major database update related process is

    running, that time archive log will be generated rapidly. In such situation if archive log

    backup is not taken at regular interval, then the archive volume will be filled up and archive

    stuck situation arise.

    Action to be taken when archive stuck situation arise

    For Unix System:

    1. Login with ora user in system

    2. Find out the location of archive log files. Generally it is /oracle//oraarch orsaparch volume.

    How to find out the location of archive log file

    You will find arch1_xxxx.dbf file in archive directory (For Example:

    SRQarch1_2833.dbf file is reside in /oracle/SRQ/oraarch volume)

    3. Check for how much free space is available in oraarch directory#bdf | grep oraarch

  • 8/2/2019 Achive Stuck Situation and Troubleshooting

    2/5

    /dev/vg12/oraarch 87785472 87785472 0 100% /oracle/SRQ/oraarch

    In above example oraarch directory is 100% utilized. We need to move files from oraarch

    volume to other directory.

    4. Immediately create ticket with concern database backup teamto take thearchive log backup of system for until further notice.

    5. Find out the free volume in system where you can move the archive log file. Thevolume must be from sapdata volume, where you can copy the archive log files.

    How to find out free volume in system:

    srqadm@hg20449:/ > bdf | grep sapdata

    Filesystem kbytes used avail %used Mounted on

    /dev/vg11/sapdata9 52428800 40963976 11375328 78% /oracle/SRQ/sapdata9

    /dev/vg11/sapdata8 52428800 40963976 11375328 78% /oracle/SRQ/sapdata8

    /dev/vg11/sapdata7 52428800 40963976 11375328 78% /oracle/SRQ/sapdata7

    /dev/vg11/sapdata6 52428800 40963976 11375328 78% /oracle/SRQ/sapdata6

    /dev/vg11/sapdata5 57671680 55300216 2353008 96% /oracle/SRQ/sapdata5

    /dev/vg11/sapdata4 52428800 40963976 11375328 78% /oracle/SRQ/sapdata4

    /dev/vg11/sapdata3 52428800 40963976 11375328 78% /oracle/SRQ/sapdata3

    /dev/vg10/sapdat20 52428800 51204024 1215280 98% /oracle/SRQ/sapdata20

    /dev/vg11/sapdata2 52428800 40963976 11375328 78% /oracle/SRQ/sapdata2

    /dev/vg10/sapdat19 52428800 40963976 11375328 78% /oracle/SRQ/sapdata19

    /dev/vg10/sapdat18 52428800 40963976 11375328 78% /oracle/SRQ/sapdata18

    /dev/vg10/sapdat17 52428800 40963976 11375328 78% /oracle/SRQ/sapdata17

  • 8/2/2019 Achive Stuck Situation and Troubleshooting

    3/5

    /dev/vg11/sapdat16 104857600 75781696 28848816 72% /oracle/SRQ/sapdata16

    /dev/vg11/sapdat15 104857600 83973664 20720856 80% /oracle/SRQ/sapdata15

    /dev/vg11/sapdat14 52428800 40963976 11375328 78% /oracle/SRQ/sapdata14

    /dev/vg11/sapdat13 57671680 40964136 16577088 71% /oracle/SRQ/sapdata13

    /dev/vg11/sapdat12 52428800 42486840 9864360 81% /oracle/SRQ/sapdata12

    /dev/vg11/sapdat11 57671680 56324184 1337040 98% /oracle/SRQ/sapdata11

    /dev/vg11/sapdat10 52428800 50180024 2231280 96% /oracle/SRQ/sapdata10

    /dev/vg11/sapdata1 52428800 43020008 9335360 82% /oracle/SRQ/sapdata1

    In above output/oracle/SRQ/sapdata16 is having the maximum free space available

    upto 28 Gb, here you can move upto 24 Gb of archive log files from /oracle/SRQ/oraarch

    volume.

    6. Create one directory with name movedarchfiles into /oracle/SRQ/sapdata16 directory

    #cd /oracle/SRQ/sapdata16

    #mkdir movedarchfiles

    7. Copy *.dbf file from /oracle/SRQ/oraarch directory to the/oracle/SRQ/sapdata16/movedarchfiles directory

    #cd /oracle/SRQ/sapdata16/movedarchfiles

    #/oracle/SRQ/sapdata16/movedarchfiles > mv /oracle/SRQ/oraarch/*.dbf.

    Caution: Do not move all *.dbf file together, it will fill up the sapdata volume if the total

    size of all dbf file is greater than available size in sapdata volume, sapdata volume will be

    100% utilized. Only move 15 to 20 dbf file to the sapdata volume in one single action.

    Note: Only move the file upto that level by which oraarch directory usage comesbelow threshold volume.

  • 8/2/2019 Achive Stuck Situation and Troubleshooting

    4/5

    8. After moving archive log file to the above volume, go to /oracle/SRQ/saparch volumeand rename the arch.log file to arch.log.old file as per following.

    Note: arch.log file can be in saparch or oraarch directory.

    #cd /oracle/SRQ/saparch

    # /oracle/SRQ/saparch > mv arch.log arch.log.old

    9. Keep an eye on archive log backup process and be in touch with database backupteam in order to take continuous archive log backup.

    10.Once archive directory usage come down to 20% to 30%, start moving the archivefiles from the /oracle/SRQ/sapdata16/movedarchfiles to /oracle/SRQ/oraarch

    directory with following command

    # mv /oracle/SRQ/sapdata16/movedarchfiles/*.dbf /oracle/SRQ/oraarch/.

    Note: oraarch directory should note be utilized above threshold value after moving

    the archive log file, if it is so, then only move some files and then get archive log

    backup and then once again move some files.

    11.After moving archive log file to the above volume, go to /oracle/SRQ/saparch volumeand rename the arch.log file to arch.log.old1 file as per following.

    Note: This action is required at each and every movement action of archive log files.

    arch.log file can be in saparch or oraarch directory.

    # /oracle/SRQ/saparch > mv arch.log

    arch.log.old1

  • 8/2/2019 Achive Stuck Situation and Troubleshooting

    5/5

    12.Login at R3.

    13.After successful completion of step 1 to 12, there should not be single archive file inmovedarchfiles directory and oraarch directory must be below threshold value.

    14.After successful following step 1 to 13, delete the movedarchfiles directory fromsapdata volume

    #cd /oracle/SRQ/sapdata16

    #rm /oracle/SRQ/sapdata16/movedarchfiles

    Process:

    1. When you get alert for archive stuck situation, update the alert with all details2. When you get CSS for archive stuck situation, update the CSS with all details.

    3