Industrial Big Data Engineer: Real-Time Analytics & Pipelines
Posted Updated
Description
INTECH AutomationIntelligence is seeking a Big Data Engineer to join our team and drive end-to-end analytics across Oil and Gas, Petrochemicals, Utilities and Manufacturing. You will design scalable ETL pipelines, ingest industrial sensor data, and deliver insights that boost safety and efficiency.
You’ll optimize the Cloudera/Hadoop stack, build real-time dashboards in Power BI and Tableau, and collaborate on OSI PI integrations, translating data into clear visuals for diverse stakeholders
Design, develop, and maintain robust ETL/ELT pipelines for pre-production and production. Ingest data from industrial sensors and enterprise sources. Implement data integration and transformation logic using PySpark, Scala, Sqoop, NiFi in Cloudera/Hadoop. Manage and schedule data workflows with Oozie or similar tools. Optimize storage and query performance with Hive, Impala, and HDFS. Develop real-time dashboards in Power BI and Tableau for KPIs. Translate complex datasets into user-friendly visuals for stakeholders. Lead dashboards for real-time monitoring and safety/performance use cases. Support predictive models by engineering necessary data features. Perform data analysis and ad-hoc reporting for manufacturing, energy, safety insights. Ensure data accuracy and reliability across reporting platforms. Assist in maintenance, configuration, and performance tuning of Cloudera and BI servers. Collaborate on OSI PI System integration with the data lake. Bachelor's or Master's degree in Computer Science, Electrical Engineering, Telecommunication, or a related field. 5+ years of experience in data engineering, big data development, or BI. Strong hands-on experience with the Apache Hadoop ecosystem (Cloudera preferred): HDFS, Hive, Impala, Oozie. Expert PySpark/Scala for building data pipelines; real-time ingestion tools like NiFi; data migration with Sqoop. Strong SQL with relational databases (SQL Server, Oracle, IBM DB2, AWS RDS). Power BI and Tableau dashboards, server administration, performance tuning. Python for data analysis and scripting. Cloud AWS experience is a plus. Familiarity with OSI PI System for sensor data integration is beneficial. Global Exposure Competitive Compensation Annual Bonus Health & Wellness Learning Opportunities Work-Life Balance Inclusive EnvironmentSkills
Sign in to read and join the discussion.