How to create a large file using shell script, To test with Hadoop Map Reduce processing

1. Open new Terminal.


2. Create 'myscript.sh'  file using below command:

touch myscipt.sh


3. Create 'smallfile.txt' file using below command:

touch smallfile.txt


4. Open the file using below command:

gedit smallfile.txt


5. Enter some sample data into the file.

"
I am going to hyderabad
I am learning hadoop course
"

6. Save the file using command: Ctrl + S


7. Now prepare a large file using below steps.


8. Open  the file using below command:

gedit myscript.sh


9.  Add the below script into the file.

#!/bin/bash
for i in {1..1000}
do
   cat smallfile.txt >> largefile.txt
done


10. Save the file using command: Ctrl + S


11. Provide execute permission to script file using below command:

chmod +x myscript.sh


12. Run the script file to generate a large file using below command:

./myscript.sh


13. Verify the 'largefile.txt' in current folder.


14. Run the above command multiple times to get more large file.


15. Try "hadoop fs -put " commad to place into HDFS.


16. Run Mapreduce programs using above file in step15.

Create a new user in MYSQL and provide PRIVILEGES to new user

1. Install mysql in ubuntu using below command:

sudo apt-get install mysql-server mysql-client

2. Connect to mysql using below command:

mysql -u root -p

provide mysql password given as part of installation step1


3. Create a new user in mysql with password

CREATE USER 'username'@'hostname' IDENTIFIED BY 'password';


4. Revoke all privileges to new user

REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'username'@'hostname';


5. Grant required privileges to new user to specific database

GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON databasename.* TO 'username'@'hostname';

6. Grant required privileges to new user to all databases

GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON *.* TO 'username'@'hostname';


7. Commit the changes

FLUSH PRIVILEGES;


8. Exit from mysql

quit;


9. Test the above steps are working or not using below command:

mysql -u username -p -h hostname

provide the 'password'

10. After verification , exit from mysql

quit;