Home / server / Questions / 1096110
In Process
King David
Asked: 2022-03-15 07:11:46 +0800 CST

hdfs + why not remove the corrupted blocks by automatic approach as cron job

  • 772

we have Hadoop cluster with 376 data-node machines ( based on HDP version - 2.6.4 )

from time to time we can see XX number of corrupted blocks

and to fix it we are performing the following:

su hdfs
hdfs fsck / -delete

since this happened from time to time

we are thinking to perform the hdfs fsck / -delete by Cron job , for example cron job can run every 00:00 , and delete the corrupted blocks

but the Question is - what are the risks when doing this?

linux hadoop hdfs

0 Answers