Wednesday, October 23, 2013

NETAPP - How to enable the autosize and autogrow on NetApp.

                      NETAPP - How to enable the autosize and autogrow on NetApp.

With this option the volume will grow with a specified space.
The first thing would be to enable the autosize option on the volume.
That can be achieved by running the following command.

NTPboxer*> vol autosize test
Volume autosize is currently ON for volume 'test'.
The volume is set to grow to a maximum of 1258080 KB, in increments of 52420 KB.

NTPboxer*> priv set diag

NTPboxer*> printflag wafl_reclaim_threshold_t
wafl_reclaim_threshold_t = 0x55

THE COMMAND ABOVE STATES THAT WHEN THE VOLUME REACHED 85% (0X55 - HEX FIGURE) IT WILL AUTOGROW BY CERTAIN FRAGMENTS.

NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/            996148KB      188KB   995960KB       0%  /vol/test/
/vol/test/.snapshot    52428KB        0KB    52428KB       0%  /vol/test/.snapshot

With the volume in place i started to dump some data into it to check the autogrow process.

NTPboxer*>
NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/            996148KB   205664KB   790484KB      21%  /vol/test/
/vol/test/.snapshot    52428KB        0KB    52428KB       0%  /vol/test/.snapshot
NTPboxer*> df -k testTue Oct 22 17:01:40 IST [NTPboxer:wafl.vol.full:notice]: Insufficient space on volume test to perform operation. 444MB was requested but only 412MB was available.


AFTER 30 SEcs

NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/            996148KB   705192KB   290956KB      71%  /vol/test/
/vol/test/.snapshot    52428KB        0KB    52428KB       0%  /vol/test/.snapshot


AFTER 20 SECS

NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/            996148KB   867568KB   128580KB      87%  /vol/test/
/vol/test/.snapshot    52428KB        0KB    52428KB       0%  /vol/test/.snapshot
NTPboxer*> Tue Oct 22 17:04:23 IST [NTPboxer:nbt.nbns.registrationComplete:info]: NBT: All CIFS name registrations have completed for the local server.
Tue Oct 22 17:04:29 IST [NTPboxer:wafl.vol.autoSize.done:info]: Automatic increase size of volume 'test' by 52420 kbytes done.

IF YOU SEE ABOVE THE VOLUME GOT AUTOSIZED WHEN IT REACHED 85%.

NTPboxer*>
NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/           1045948KB   888824KB   157124KB      85%  /vol/test/
/vol/test/.snapshot    55048KB        0KB    55048KB       0%  /vol/test/.snapshot


ADDED another 100mb

NTPboxer*>
NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/           1145548KB   959340KB   186208KB      84%  /vol/test/
/vol/test/.snapshot    60288KB        0KB    60288KB       0%  /vol/test/.snapshot

NTPboxer*>
NTPboxer*> df -k test
Filesystem               total       used      avail capacity  Mounted on
/vol/test/           1195176KB  1074676KB   120500KB      90%  /vol/test/
/vol/test/.snapshot    62904KB        0KB    62904KB       0%  /vol/test/.snapshot
NTPboxer*> Tue Oct 22 17:12:55 IST [NTPboxer:wafl.vol.autoSize.done:info]: Automatic increase size of volume 'test' by 52420 kbytes done.

IF YOU SEE ABOVE THE VOLUME GOT AUTOSIZED.

Monday, October 21, 2013

Netbackup - How to change the expiration date?

                                Netbackup - How to change the expiration date?

-------------------------------------------------------------------------------------------------------
People say that when you run this command (the command below) you get all the required information with regards to the media which was used for the backup.

/usr/openv/netbackup/bin/admincmd/bpimagelist -client [client] -d [date of the backup] -e [1 day after the backup]

BUT UNFORTUNATELY --

It doesn’t give the backup id of which the expiration date needs to be changed.

So this is what i tried at my end -

I ran

# ./bpimagelist -client unixfoo90 -media -d 04/23/2010 -e 04/24/2010
A00461 ajbkup 1272114266 17 81416886 10
A00326 ajbkup 1272114266 43 69301582 10
A51671 ajbkup 1272114266 14 19195648 10
A50155 ajbkup 1272114266 41 56309070 10
A51064 ajbkup 1272080168 5 75213750 10

This gave me the time in ctime.

I converted the ctime into the normal time.

# ./bpdbm -ctime 1272080168
1272080168 = Fri Apr 23 22:36:08 2010

I noted down the media ID for the backup which happened on - Apr 23 22:36:08 2010 --which matched with the request.

With this I ran..

# ./bpimmedia -L -mediaid A51064 -client unixfoo90

Backup-ID            Policy     Type  RL  Files   C  E  T  PC  Expires
Copy Frag    KB Type Density FNum      Off       Host       DWO MPX Expires          RL MediaID
------------------------------------------------------------------------------------------------

unixfoo90_1272247531  sapdistdb  INCR  10   16      N  N  R  1   21:05 06/13/2010
 1   1        0 RMed hcart3  32        2173295   ajbkup   901 Y   21:05 06/13/2010 10 A00238
 1   2      224 RMed hcart3  33        2173305   ajbkup   901                         A00238
 2   1        0 RMed hcart3  161       3473274   ajbkup   124 Y   21:05 06/13/2010 10 A51064
 2   2      224 RMed hcart3  162       3473284   ajbkup   124                         A51064

unixfoo90_1272080168  sapdistdb  INCR  10   26      N  N  R  1   22:36 06/11/2010
 1   1        0 RMed hcart3  156       3798984   ajbkup   107 Y   22:36 06/11/2010 10 A00461
 1   2     1024 RMed hcart3  157       3798990   ajbkup   107                         A00461
 1   3  1921024 RMed hcart3  158       3799006   ajbkup   107                         A00461
 1   4 73037824 RMed hcart3  159       813953   ajbkup   107                         A00461
 1   5   253878 RMed hcart3  160       4099259   ajbkup   107                         A00461
 2   1        0 RMed hcart3  134       1427550   ajbkup   124 Y   22:36 06/11/2010 10 A51064
 2   2     1024 RMed hcart3  135       1427556   ajbkup   124                         A51064
 2   3  1921024 RMed hcart3  136       1427572   ajbkup   124                         A51064
 2   4 73037824 RMed hcart3  137       1442519   ajbkup   124                         A51064
 2   5   253878 RMed hcart3  138       1727825   ajbkup   124                         A51064

unixfoo90_1271992736  sapdistdb  INCR  10   18      N  N  R  1   22:18 06/10/2010
 1   1      768 RMed hcart3  117       3021213   ajbkup   111 Y   22:18 06/10/2010 10 A00461
 1   2    69321 RMed hcart3  118       3021242   ajbkup   111                         A00461
 2   1      768 RMed hcart3  96        671044    ajbkup   124 Y   22:18 06/10/2010 10 A51064
 2   2    69321 RMed hcart3  97        671073    ajbkup   124                         A51064

Finally I got the BACKUP ID -- unixfoo90_1272080168

Later I found that I could have easily got this with the below mentioned command –

# ./bpimagelist -L -client unixfoo90 -d 04/23/2010 |grep "Backup ID"
Backup ID:         unixfoo90_1272718818
Backup ID:         unixfoo90_1272684933
Backup ID:         unixfoo90_1272680883
Backup ID:         unixfoo90_1272596918
Backup ID:         unixfoo90_1272519172
Backup ID:         unixfoo90_1272425403
Backup ID:         unixfoo90_1272339748
Backup ID:         unixfoo90_1272247531
Backup ID:         unixfoo90_1272114266
Backup ID:         unixfoo90_1272080168

Later,

# ./bpexpdate -recalculate -backupid unixfoo90_1272080168 -d 10/23/2010 12:00:00

VERIFICATION --

# ./bpimagelist -client unixfoo90 -d 04/23/2010 -e 04/24/2010 -U
Backed Up         Expires       Files       KB  C  Sched Type   Policy
----------------  ---------- -------- --------  -  ------------ ------------
04/24/2010 08:04  06/12/2010      156 75504718  N  Full Backup  bkupatp

04/23/2010 22:36  10/23/2010       26 75213750  N  Differential bkupatp

Tuesday, October 8, 2013

VMWARE - I/O error - EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block

VMWARE - I/O error - EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block

We have a ESX environment wherein we have a linux server which has iSCSI storage attached to it.
Since the iSCSI from the NetApp storage is mapped to the linux VM it doesnt find itself part of the datastore.
For some reason the linux OS stopped responding to the 1 TB iSCSI LUN.
I unmapped the LUN from the igroup and mapped it back.
The problem didnt go away.
On checking the logs of the server /var/log/messages
I found the following.
Even the reinitialize didnt do any wonder.
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2063176208
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1971791144
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849697344
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849697440
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849697456
Oct  7 13:02:24 AF$$^&rod1 kernel: EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block - inode=57820282, block=231212135
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2345037096
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2111523576
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2332778144
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2511344960
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849695496
Oct  7 13:02:24 AF$$^&rod1 kernel: EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block - inode=57816881, block=231211923
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849695496
Oct  7 13:02:24 AF$$^&rod1 kernel: EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block - inode=57816881, block=231211923
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849695496
Oct  7 13:02:24 AF$$^&rod1 kernel: EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block - inode=57816881, block=231211923
Oct  7 13:02:24 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 1849695496
Oct  7 13:02:24 AF$$^&rod1 kernel: EXT4-fs error (device dm-8): __ext4_get_inode_loc: unable to read inode block - inode=57816881, block=231211923
Oct  7 13:02:29 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2308784176
Oct  7 13:02:29 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2308784176
Oct  7 13:02:29 AF$$^&rod1 kernel: end_request: I/O error, dev dm-7, sector 2308784176
Oct 7 13:36:19 AF$$^&rod1 mcelog: failed to prefill DIMM database from DMI data


This mostly being the I/O error there was no away the OS was getting away with the iSCSI LUN.
The only way we got rid of this error was by rebooting the machine.
This was like the last resort for us.
Thank you..

Monday, October 7, 2013

BACKUP - NetBackup - How to run and collect the DataCollect Logs

               BACKUP - NetBackup - How to run and collect the DataCollect Logs


      Login to the appliance CLISH.Once logged in youi will see the following MENU



At the prompt type the option “Support”




Back on the prompt type the second option from the menu “DataCollect”.

This would start the processing the DatCollect command.
All the DataCollect lo]gs will be stored in the temp directory.
You can open a share on the appliance to access those logs.


Thank you...

Friday, October 4, 2013

STORAGE - NetApp - Adding a ndmpuser in NetApp for NDMP backups

NetApp - Adding a ndmpuser in NetApp for  NDMP backups

A ndmpuser can be added to the storage system to make the NDMP backup running.
This ndmpuser can be added to the credentials of the backup software ( NetBackup / Networker).

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
NetApp1> useradmin user add ndmpuser -g "Backup Operators"
New password:
Retype new password:
User <ndmpuser> added.
NetApp1> Mon Apr  8 17:25:40 CST [NetApp1:useradmin.added.deleted:info]: The user 'ndmpuser' has been added.

NetApp1>
NetApp1> ndmpd password ndmpuser
password QKxD########dmL


Thank you..

STORAGE - NetApp - Tracking NFS Issues

                          NetApp - Tracking NFS issues


Many a times you would have experienced a lot of problems while mounting a NFS volume on any host from a NetApp Storage.
Most of the times we end up checking the log files on the host side thinking that the issue might be with the host itself.

Recently i ran into an issue wherein i was not able to mount a NFS volume from my NetApp controller to the Linux Server.

A couple of thumb rules that you need to follow when you doing a task like this are -

1. The NFS versions on both Storage (NetApp) and the HOST (Solaris /Linux/etc etc).
2. The connectivity between the Storage appliance and the HOST.

If still the issue persists --

Do the following on the Storage appaliance.

At the prompt of the NetApp storage appliance do the following -

NetApp1> options nfs.mountd.trace
nfs.mountd.trace             off
NetApp1> options nfs.mountd.trace on

Turning ON this option on the storage appliance will tell you where exactly the problem is.

Hope this helps.

Thank you..

STORAGE - NetApp - Unable to assign serial number to LUN

         STORAGE - NetApp - Unable to assign serial number to LUN


While attaching the LUN to an igroup which has the host iscsi initiator target set sometimes you might run into a problem wherein the LUN doesnt get attached to the igroup hence not allocating the block based space to the host.

This can be rectified by setting up the counter on your NetApp system --

On your Filer head
Run the following command -

# priv set diag
# lun sn-counter show

This will shoe the current value of the counter.
This would display the countyer set for generating the serial numbers.

#lun sn-counter set -t

This will reset the counter to default.

Once done try to allocate the LUN once again and you should be good to go.

Thank you..

Friday, September 20, 2013

Hello Engineer

Still a student in this big world of Storage ,backup and Virtualization every information gathered gets  archived forever with a retention level of infinity.
One of the most important domains in any IT or manufacturing sector StoRage/Backup/Virtualization go hand in hand.

What you have now is what we had maintained yesterday.

The intent of this blog is to share my technical attributes and troubleshooting processes and problems that I faced in the domains that I work.

This would not only help me but also other people who work on these domains.

Please do write to me if you find theres a better way to deal with any of the topics.

Have a wonderful day.

Ajay Warrier