Experienced IT professional with total work experience of 3+ years in the field of Big data and software development . I am currently pursuing MS program in Management Information System at State University of New York at Buffalo, New York.
WHAT I CAN OFFER:
• Hands on Work experience with Big data ecosystem and it’s various components such as HDFS, Hive , Pig , Scoop , MapReduce , Spark(pySpark).
• Good understanding/knowledge of Hadoop architecture and its components .
• Extensive working experience with programming languages such as Python and Java.
• Good working knowledge of SQL .
• Experience on working with BI Reporting tool such as Tableau.
• Hands on experience with AWS services such as EC2,S3,RDS, Redshift.
• Comprehensive hands on experience in Software Development .
• Hands on experience in developing RESTful API’s using Java, J2EE, Spring MVC, Hibernate .
• Good Knowledge of Software Development Life Cycle (SDLC), Agile Methodologies.
• Holds experience of directly working with client from requirement gathering to delivering deliverables.
CORE COMPETENCIES:
Big Data: Hadoop, Map Reduce, Hive, Pig, Sqoop, Flume, HBase , SQL, Spark(pySpark)
Web Development: J2EE, Servlets, Spring Framework, Hibernate
Database: Oracle SQL, My SQL
Programming Languages: Python, Java, C++
Cloud : AWS
IDE: Eclipse, Intellij Idea, Spyder, Jupyter
Servers: Tomcat, Jetty, Weblogic, Websphere
BI Reporting Tool: Tableau , Excel
Web development Build Tools: Git, Gerrit ,Sonar ,Jenkins , Maven ,SVN,
Incident Management Tools: Service Now, ALM
Operating System : Linux,Windows
ONLINE PRESENCE:
• Github: https://github.com/Dipankar17
• Tableau: https://public.tableau.com/profile/dipankarsharma
ROLES THAT MATCH MY EXPERTISE INCLUDE:
• Big Data Engineer, Data Engineer, Software Developer,Systems Engineer, Programmer Analyst, Analyst, Senior Analyst, Software Engineer
CONTACT INFORMATION:
If my profile interests you , please send me connection request or feel free to reach out to me at dipankar178@gmail.com
-
Experience
Project: Data Ingestion
Role: Big data Developer
• Ingested e-statement data of last ten years for banking client from traditional database into HDFS and Hive tables using Sqoop.
• Formulated optimized Hive queries to process the mortgage data and helped bank in identifying the customers with occurrences of late repayment history.
• Implemented market segment analysis on credit card data to identify the credit card usage in different age groups and visualized the data using Tableau.
• Developed Java MapReduce code to analyze server’s CPU and memory utilization logs which helped client to understand current system’s efficiency.
Project: Building Digital Statement Platform Version 2.0
Role: Java Developer
• Worked on developing Java REST APIs for banking website to replace the existing statement services, which improved overall performance of the website.
• Designed testing scenarios and wrote Junit test cases for unit testing to validate the possible outcomes.
• Investigated and resolved code defects that occurred during system integration testing and user acceptance testing.
• Acquired knowledge of Software Development Life Cycle (SDLC); Agile and Waterfall Methodologies.
• Collaborated directly with client from requirement gathering to delivering deliverables.
-
Projects
Data Migration to AWS Redshift:
• Used Identity and Access Management (IAM) service for creating read only role for redshift to access data files from S3 bucket.
• Set up AWS Redshift multi node cluster, integrated the cluster with SQL workbench and designed data models to store data.
• Designed data pipeline to migrate data stored in Amazon S3 bucket to tables created in redshift database.
Find Non-English text words from folk songs files stored in HDFS:
• Transferred more than 100 folk song files stored in local system to HDFS.
• Developed Map Reduce Program in python for filtering Non-English words from the data files stored in HDFS.
Predict Best Playing XI Football Team:
• Cleaned, Normalized and Segregated data into multiple categories and stored data in different pandas data frames.
• Performed data analysis on the segregated data to understand the pattern of attributes of players for different positions along with dependency of these attributes on different player characteristics such as BMI, age, wage, etc. and depicted the visualizations using seaborn and matplotlib libraries.
• Predicted best possible team for 3-4-3 formation by feeding data of 20K players into K – Nearest Neighbor (KNN) model.
• Visualized final predicted team on football field using Python Image Processing module PIL.