Thursday, January 30, 2020

Data Platform Tips 64 - Recommended node configuration and VM sizes for HDInsight Clusters

The default and minimum recommended node configuration and virtual machine sizes for Azure HDInsight Clusters are listed below.

Size
vCPU
Memory: GiB
Temp storage (SSD) GiB
Max temp storage throughput: IOPS / Read MBps / Write MBps
Max data disks / throughput: IOPS
Max NICs / Expected network bandwidth (Mbps)
Standard_D3_v2
4
14
200
12000 / 187 / 93
16 / 16x500
4 / 3000
Standard_D4_v2
8
28
400
24000 / 375 / 187
32 / 32x500
8 / 6000
Standard_D5_v2
16
56
800
48000 / 750 / 375
64 / 64x500
8 / 12000
Standard_D12_v2
4
28
200
12000 / 187 / 93
16 / 16x500
4 / 3000
Standard_D13_v2
8
56
400
24000 / 375 / 187
32 / 32x500
8 / 6000
Standard_D14_v2
16
112
800
48000 / 750 / 375
64 / 64x500
8 / 12000
Standard_A1_v2
1        
2          
10            
1000 / 20 / 10                                          
2 / 2x500              
2 / 250                
Standard_A2_v2
2
4
20
2000 / 40 / 20
4 / 4x500
2 / 500
Standard_A4_v2
4
8
40
4000 / 80 / 40
8 / 8x500
4 / 1000

If you need more than 32 worker nodes in a cluster, select a head node size with at least 8 cores and 14 GB of RAM.

The only cluster types that have data disks are Kafka and HBase clusters with the Accelerated Writes feature enabled.

No comments:

Post a Comment