[NetApp] 2 nodes upper Data lif happen redundant migrate other node not work?

在一般Case案件大多安裝是一套NA,於同事發生一件案例是二套NA FAS8200等於有 4 nodes,在驗證Data Lifs過程會發生lif移到線路down port造成問題。

Sample : node1 & node 2 (pairs) ; node3 & node4 (pairs)

node1 – 拔除nic cable ,lif migrate to node3

node2 – 拔除nic cable , lif migrate to node1 (馬上GG,因為node1線路全部拔除)

Resolution :

::> net int modify -vserver {SVM} -lif {lif-name} -failover-policy broadcast-domain-wide // 預設 SVM Data Lif 都用 system-defined

** 共有五種 Failover policy **

  • broadcast-domain-wide :
    This is the default setting for the cluster management LIF.You would not want to assign this policy to node management LIFs or cluster LIFs because in those cases the ports must be on the same node.
  • system-defined :
    This is the default setting for data LIFs.This setting enables you to keep two active data connections from two unique nodes when performing software updates. It allows for rolling upgrades; rebooting either odd-numbered or even-numbered nodes at the same time.
  • local-only:
    This is the default setting for cluster LIFs and node management LIFs.

    This value cannot be changed for cluster LIFs.

  • sfo-partner-only :
    Only those ports in the failover group that are on the LIF’s home node and its SFO (storage failover) partner node.
  • disabled:
    The LIF is not configured for failover.


    Reference :
    NetApp – Types of failover policies

 

 

[Storage] 增加NetApp DOT 7.x / 8.x / 9.x Filer被ping (ICMP Packet) 數量

因客戶需求有APP需每秒輸送出1000個ICMP packet 數量來判別Filer是否存在;因為原廠預設是針對Client單一能每秒150 ICMP packets來防止DoS ( denial-of-service) 攻擊.因此需提升接受單一Client能每秒1000個ICMP Packet.

<< Resolution >>

7.x~8.x

[7 mode]
options ip.ping_throttle.drop_level <數量> // default 150 ; Maximum 4294967295 (42億多)

[Clustered mode]
<ONTAP 8.x>
::> system run -node {nodename} -command “options ip.ping_throttle.drop_level <數量>"
<ONTAP 9.x>
system run -node {nodename} -command “options ip.ping_throttle.drop.level  <數量>

假若要不設限可以設為 ‘0’
<ONTAP 8.x>
system run -node {nodename} -command “options ip.ping_throttle.drop_level 0

<ONTAP 9.x>
system run -node {nodename} -command “options ip.ping_throttle.drop.level 0"

Checking the ping throttling threshold status
::> netstat -p icmp

 


Reference :

1. NetApp – Increasing the ping throttling threshold value

2. NetApp Document ID : FA1394

[Storage] HDS 存儲設備常見縮寫 / 預設帳號/密碼

  • DKC (Disk Controller)
  • DKU (Disk Unit)
  • DKA (Disk Adapter)         // Back-end , 連接內部硬碟
  • CHA (Channel Adapter)  // Front-end , 連接外部主機
  • SM (Share Memory)
  • CM (Cache Memory)
  • CP (PCB) // 處理器主板
  • SVP (Service Processor) // 系統控制台;用於設備的管理
  • HDU (Hard Disk Unit)
  • HDD (Hard Disk Driver)

  • VSP  maintenance / raid-mainte 或 etniam-diar , raid-maintenance
  • VSP G1000 , maintenance / raid-maintenance
  • USP Administrator
  • SVP / raid-login

  • Hi-Track , administrator / hds

[NetApp] ESXi NFS use Thin Provisioning

—–Pre-check—–
* NFSv3 must be enabled on the storage system
* NFSv4.1 is available only on ONTAP 9.0

* VMware vSphere 5.0 or later must be available
—–End————–

1.download NetApp VAAI Plug-in ; 載點 https://nt-ap.com/2HxiF4T

2.install NetApp VAAI Plug-In @ESXi
> esxcli software vib install -n NetAppNasPlugin -d /NetAppNasPlugin.zip

3.@NetApp type command
<clustered-mode>
::> vserver nfs modify –vserver {SVM-name} -vstorage enabled
<7-mode>
> options nfs.vstorage.enable on
<7-Mode CLI for vFiler units>
> vfiler run vfiler_name options nfs.vstorage.enable on

4. verify install state
> esxcli software vib list | grep -i netapp

5. verify vaai enable (value是否為 1 (enable);若否請到 6. )
> esxcfg-advcfg -g /DataMover/HardwareAcceleratedMove
> esxcfg-advcfg -g /DataMover/HardwareAcceleratedInit

6. enable vaai
> esxcfg-advcfg -s 1 /DataMover/HardwareAcceleratedInit
> esxcfg-advcfg -s 1 /DataMover/HardwareAcceleratedMove

7.(options verify)
> vmkfstools -Ph /vmfs/volumes/onc_src/
sample
> mkfstools -Ph /vmfs/volumes/46db973f-cca15877

 

[Linux] RHEL6 install Hitachi multipath ‘HDLM’

step01. 準備好 HDLM Linux License *.plk ; 可以查看確認該檔文字內容有無 for Linux ,若沒有可不是隨便將for Windows 改為 for Linux就可以用唷!

step02. 更名License KEY改為或複製成hdlm_license
# mv “HDS VSP GXXX SNXXXXXX DXXX XXXXXXXXXXXX".plk /var/tmp/hdlm_license

step03.  放入 HDLM CD-ROM

step04. 掛載HDLM光碟
# mount -t iso9660 /dev/cdrom /media
Or
# mount /dev/cdrom /media

step05. check HDLM version
# /media/HDLM_Linux/installhdlm -v

step06. 安裝HDLM
# /media/HDLM_Linux/installhdlm
—————————————————–
KAPL09210-I installhdlm will now start.
KAPL09093-I HDLM 8.5.1-00 will be installed. Is this OK ? [y/n]: y
Preparing packages for installation…
KAPL09076-I The permanent license was installed.
HDLM-8.5.1.0.808-4
KAPL09043-I The installation of HDLM-8.5.1.0.808-4 completed successfully.
KAPL09211-I installhdlm completed successfully.
—————————————————–

step07. 檢查HDLM已安裝
# rpm -qa | grep -i hdlm

step08. 詳看HDLM資訊
# rpm -qi HDLM

step09. 若未重開機啟動
# /opt/DynamicLinkManager/bin/dlmstart
# /etc/init.d/DLMManager start

step10. 加入HDLM環境變數 > /opt/DynamicLinkManager/bin
[BourneAgain shell ]
# PATH=$PATH:/opt/DynamicLinkManager/bin ; export PATH

[C shell]
# set path= ( $path /opt/DynamicLinkManager/bin )

step11. 若未重啟系統請立即重啟一次
# shutdown -r now
Or
# init 6

step12. 查看dlnkmgr運作狀態及設定,版本,Load Balance模式,自動容錯切換狀態(aft),硬碟
# /opt/DynamicLinkManager/bin/dlnkmgr view -sys
我大多會注意三項資訊及狀態 > { Load Balance , Path Health Checking , Auto Failback )

HDLM-view-sys.png# /opt/DynamicLinkManager/bin/dlnkmgr view -lu
# /opt/DynamicLinkManager/bin/dlnkmgr view -path
# /opt/DynamicLinkManager/bin/dlnkmgr view -drv

Done.

–Maintenance–

**啟用Load Balance及變更模式為Round-Robin**
# /opt/DynamicLinkManager/bin/dlnkmgr set -lb on -lbtype rr
(模式如下)
rr       >for the Round Robin algorithm
exrr   >for the Extended Round Robin algorithm
lio      >for the Least I/Os algorithm
exlio  >for the Extended Least I/Os algorithm
lbk     >for the Least Blocks. algorithm
exlbk >for the Extended Least Blocks algorithm

**啟用及變更Path Health Checking及間隔時間**
# /opt/DynamicLinkManager/bin/dlnkmgr set -pchk on -intvl 10

**啟用Automatic Failback及間隔時間**
# /opt/DynamicLinkManager/bin/dlnkmgr set -afb on -intvl 10 (預設是關閉)

**查詢HDLM指令用法**
# /opt/DynamicLinkManager/bin/dlnkmgr help view


Reference :

1. Eddynurh’s Weblog – HDLM Installation

2. 馬克思 Linux學習筆記 – HDLM install on RHEL5.2

3. (原廠) Hitachi Command Suite – Dynamic Link Manager(for Linux®) User Guide

4. 杨红1989 – LINUX 安装HDS HDML报错

[VMware] Consolidation failed for disk node ‘scsi0:0’:5 (Input/output error)

當客戶反應說他們因儲存設備某一個LUN有問題;需立即將某台虛擬機器搬離這個有問題的LUN。但卻無法搬離開且有些怪異現象出現!

狀況查看:
1.  該台VM service運作正常。
2. 但用備份方式是都有錯誤狀況. e.g.
# use export to .ova/ovf > 失敗 (Failed to eport Virtual Machine: Unable to read data from the transport connection: The connection was closed.)
# use vmkfstools clone vmdk > 失敗 ( clone: 100% done.Failed to clone disk: Input/output error (327689).)
# use Backup (HPE VM explorer) > 失敗 (Failed: <VM/VM-000005-deltavmdk cannot be downloaded. ‘HTTP/1.1500 internal Server error’)
# use VMware Data Protect backup > 失敗 (整併磁碟節點 ‘scsi0:0’ 失敗:5 (input/output error).)
# 該VM瀏覽Datastore會發現大量 vm-000001.vmdk 增長到vm-000009.vmdk(快照檔案) ; 但由snapshot manager查看是無任何快照記錄.
# vMotion 搬移失敗.
# 指令 ‘esxcli storage core path stats get‘ 查看到僅有對應有問題的LUN,有所謂(Failed Read Operations: 74 / Failed Write Opterations: 2 )

由以上種種現象來看確實Storage volume有些狀況;但FibreChannel Switch & HBA是沒什麼問題.

最後解決客戶這問題僅能有OS level來處理;我則是用Acronis 11.7 offline backup all disk (ignore error才能成功) , then Create new VM and rexovery that image .

Done.