75 lines
1.4 KiB
Markdown
75 lines
1.4 KiB
Markdown
### List of Commands
|
|
|
|
1. **Create a Directory for Your Project**:
|
|
|
|
```bash
|
|
mkdir ~/hadoop_char_count
|
|
cd ~/hadoop_char_count
|
|
```
|
|
|
|
2. **Compile the Java Files**:
|
|
|
|
```bash
|
|
javac -classpath $(hadoop classpath) -d . CharacterCountMapper.java CharacterCountReducer.java CharacterCountDriver.java
|
|
```
|
|
|
|
3. **Create the JAR File**:
|
|
|
|
```bash
|
|
jar cvf CharacterCount.jar *.class
|
|
```
|
|
|
|
4. **Create Input Directory in HDFS** (if needed):
|
|
|
|
```bash
|
|
hdfs dfs -mkdir -p /user/hduser/input
|
|
```
|
|
|
|
5. **Upload Input File to HDFS**:
|
|
|
|
```bash
|
|
hdfs dfs -put /path/to/your/local/input.txt /user/hduser/input/
|
|
```
|
|
|
|
6. **Run the MapReduce Job**:
|
|
|
|
```bash
|
|
hadoop jar CharacterCount.jar CharacterCountDriver /user/hduser/input /user/hduser/output
|
|
```
|
|
|
|
7. **Remove Existing Output Directory** (if needed):
|
|
|
|
```bash
|
|
hdfs dfs -rm -r /user/hduser/output
|
|
```
|
|
|
|
8. **List Contents of the Output Directory**:
|
|
|
|
```bash
|
|
hdfs dfs -ls /user/hduser/output
|
|
```
|
|
|
|
9. **View the Output File**:
|
|
|
|
```bash
|
|
hdfs dfs -cat /user/hduser/output/part-r-00000
|
|
```
|
|
|
|
10. **View Output with `more` or `less`**:
|
|
|
|
```bash
|
|
hdfs dfs -cat /user/hduser/output/part-r-00000 | more
|
|
```
|
|
or
|
|
```bash
|
|
hdfs dfs -cat /user/hduser/output/part-r-00000 | less
|
|
```
|
|
|
|
11. **Copy Output to Local File System (Optional)**:
|
|
|
|
```bash
|
|
hdfs dfs -get /user/hduser/output/part-r-00000 /path/to/local/directory/
|
|
```
|
|
|
|
---
|