-
Experience
I am working as Hadoop developer at present. I have developed different frameworks in technologies teradata, spark/scala, python. I have automated many tasks reducing the time to do work. I have working knowledge in technologies and databases like Teradata, Oracle, DB2, Spark, Scala, Python, Hive, Machine Learning, Pandas, Numpy, Scikit-Learn, GIT Hub. Also I have knowledge on AWS, devop tools.
-
Projects
1. Migration project from Mainframes,DB2 to Datastage, Teradata, python and unix. Created a framework in Teradata that makes everything configurable.
2. Created Spark Scala Framework that automates the running of business rules and will prepare the layer for reporting in Hive.
3. Created a application which will send the mails hourly about the status of the jobs and applications running. It can restart on some of the failures. It will send the mail with complete error details and logs.
4. Created the datastage job instead of Unix where it will send the files to the destination in 3 times check and will send N number of files.