Added code for hadoop.
This commit is contained in:
@@ -0,0 +1,74 @@
|
||||
### List of Commands
|
||||
|
||||
1. **Create a Directory for Your Project**:
|
||||
|
||||
```bash
|
||||
mkdir ~/hadoop_char_count
|
||||
cd ~/hadoop_char_count
|
||||
```
|
||||
|
||||
2. **Compile the Java Files**:
|
||||
|
||||
```bash
|
||||
javac -classpath $(hadoop classpath) -d . CharacterCountMapper.java CharacterCountReducer.java CharacterCountDriver.java
|
||||
```
|
||||
|
||||
3. **Create the JAR File**:
|
||||
|
||||
```bash
|
||||
jar cvf CharacterCount.jar *.class
|
||||
```
|
||||
|
||||
4. **Create Input Directory in HDFS** (if needed):
|
||||
|
||||
```bash
|
||||
hdfs dfs -mkdir -p /user/hduser/input
|
||||
```
|
||||
|
||||
5. **Upload Input File to HDFS**:
|
||||
|
||||
```bash
|
||||
hdfs dfs -put /path/to/your/local/input.txt /user/hduser/input/
|
||||
```
|
||||
|
||||
6. **Run the MapReduce Job**:
|
||||
|
||||
```bash
|
||||
hadoop jar CharacterCount.jar CharacterCountDriver /user/hduser/input /user/hduser/output
|
||||
```
|
||||
|
||||
7. **Remove Existing Output Directory** (if needed):
|
||||
|
||||
```bash
|
||||
hdfs dfs -rm -r /user/hduser/output
|
||||
```
|
||||
|
||||
8. **List Contents of the Output Directory**:
|
||||
|
||||
```bash
|
||||
hdfs dfs -ls /user/hduser/output
|
||||
```
|
||||
|
||||
9. **View the Output File**:
|
||||
|
||||
```bash
|
||||
hdfs dfs -cat /user/hduser/output/part-r-00000
|
||||
```
|
||||
|
||||
10. **View Output with `more` or `less`**:
|
||||
|
||||
```bash
|
||||
hdfs dfs -cat /user/hduser/output/part-r-00000 | more
|
||||
```
|
||||
or
|
||||
```bash
|
||||
hdfs dfs -cat /user/hduser/output/part-r-00000 | less
|
||||
```
|
||||
|
||||
11. **Copy Output to Local File System (Optional)**:
|
||||
|
||||
```bash
|
||||
hdfs dfs -get /user/hduser/output/part-r-00000 /path/to/local/directory/
|
||||
```
|
||||
|
||||
---
|
||||
Reference in New Issue
Block a user