> For the complete documentation index, see [llms.txt](https://docs.bdb.ai/data-pipeline/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.bdb.ai/data-pipeline/components/readers/hdfs-reader.md).

# HDFS Reader

HDFS stands for **Hadoop Distributed File System**. It is a distributed file system designed to store and manage large data sets in a reliable, fault-tolerant, and scalable way. HDFS is a core component of the Apache Hadoop ecosystem and is used by many big data applications.

This component reads the file located in HDFS(**Hadoop Distributed File System).**

All component configurations are classified broadly into 3 section

* [​Basic Information​](/data-pipeline/components/component-base-configuration.md)​
* Meta Information
* [Resource Configuration​](/data-pipeline/components/resource-configuration.md)

## **Configuring the  Meta Information tab of the HDFS Reader**

* **Host IP Address:** Enter the host IP address for HDFS.
* **Port:** Enter the Port.
* **Zone:** Enter the Zone for HDFS. Zone is a special directory whose contents will be transparently encrypted upon write and transparently decrypted upon read.
* **File Type:** Select the File Type from the drop down. The supported file types are:
  * ***CSV:*** The ***Header*** and ***Infer Schema*** fields get displayed with ***CSV*** as the selected File Type. Enable ***Header*** option to get the Header of the reading file and enable Infer Schema option to get true schema of the column in the CSV file. 
  * ***JSON***: The **Multiline** and **Charset** fields get displayed with ***JSON*** as the selected File Type. Check-in the **Multiline** option if there is any multiline string in the file.
  * ***PARQUET***: No extra field gets displayed with PARQUET as the selected File Type.
  * ***AVRO***: This File Type provides two drop-down menus.
    * ***Compression***: Select an option out of the ***Deflate*** and ***Snappy*** options.
    * ***Compression Level***: This field appears for the Deflate compression option. It provides **0** to **9** levels via a drop-down menu.
  * ***XML:*** Select this option to read XML file. If this option is selected, the following fields will get displayed:
    * **Infer schema:** Enable this option to get true schema of the column.
    * **Path:** Provide the path of the file.
    * **Root Tag:** Provide the root tag from the XML files.
    * **Row Tags:** Provide the row tags from the XML files.
    * **Join Row Tags:** Enable this option to join multiple row tags.
  * **Path:** Provide the path of the file.
  * **Partition Columns**: Provide a unique Key column name to partition data in Spark.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.bdb.ai/data-pipeline/components/readers/hdfs-reader.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.