Developer

Cognizant is hiring a Developer to design, build, and optimize PySpark-based data solutions supporting analytics and content decision-making for a global media and entertainment organization.

The role focuses on scalable data pipelines, distributed data processing, data quality, performance optimization, data integration, data modeling, monitoring, and production support for large-scale media and entertainment datasets.

Key Responsibilities

• Design robust PySpark data pipelines to ingest, transform, and aggregate large-scale datasets.
• Develop optimized PySpark code for batch and near real-time data processing.
• Collaborate with data engineers, analysts, product teams, and business stakeholders.
• Develop reusable data frameworks and components for streaming, video-on-demand, and advertising data.
• Implement data quality checks, validation routines, and monitoring mechanisms.
• Integrate data from multiple media platforms and content management systems.
• Develop unified data models for audience engagement and catalog performance.
• Optimize storage formats, partitioning strategies, and PySpark execution configurations.
• Deliver data outputs supporting programming strategy, recommendation engines, campaign measurement, and dashboards.
• Document technical designs, PySpark jobs, data flows, and configurations.
• Follow data governance, security, and privacy standards.
• Troubleshoot production pipeline issues and perform root cause analysis.
• Contribute to improvements in data pipeline architecture, coding practices, and automation.
• Support testing by preparing sample datasets and validating outputs.

Required Skills

• PySpark
• Python
• Distributed Data Processing
• Big Data
• Data Engineering
• Data Pipelines
• Data Transformation
• Data Aggregation
• Batch Processing
• Near Real-Time Data Processing
• Data Modeling
• Data Warehousing
• Data Integration
• Data Quality
• Data Validation
• Data Monitoring
• Performance Tuning
• Resource Optimization
• Job Scheduling
• Version Control
• Collaborative Development
• Troubleshooting
• Root Cause Analysis
• Data Governance
• Data Security
• Data Privacy

Domain Knowledge

• Media and Entertainment
• Audience Measurement
• Content Metadata
• Streaming Events
• Advertising Metrics
• Catalog Management
• Audience Engagement
• Content Performance

Preferred/Additional Skills

• Big Data Platforms
• Data Warehousing Technologies
• Reusable Data Frameworks
• Storage Optimization
• Partitioning Strategies
• PySpark Performance Optimization
• Data Quality Frameworks
• Monitoring Tools
• Production Data Pipeline Support
• Automation