Climate Resilience Data Platform
The Climate Resilience Data Platform is a tool designed to capture and analyze the evolving narratives around climate change. By mining data from media articles and social networks, the platform tracks how climate discussions shift over time and across different regions. This provides unique insights into public perceptions and helps identify key narrative trends.
Key Features
🌍 Geographic Insights
The platform enriches climate-related conversations with geolocation mapping, offering users a detailed look into how different regions talk about climate change.
🧠 AI-Driven Data Processing
We leverage AI agents using LangChain and OpenAI to perform complex data enrichment tasks, such as conversation classification and narrative association, enabling a nuanced understanding of climate discourse.
⚙️ Flexible Architecture
Built with Dagster at its core for orchestrating workflows, the platform is designed to scale efficiently and handle dynamic data sources like social networks and media outlets.
🔎 Public Discourse Analysis
Track online conversations about climate change, analyze the dominant narratives, and explore how public sentiment evolves over time. This helps users gain deep insights into regional and temporal patterns.
Architecture Overview
The Climate Resilience Data Platform is built on a layered, modular architecture to ensure scalability, flexibility, and performance. The core components include:
- Dagster: Orchestrates workflows and data pipelines, ensuring they run smoothly and efficiently.
- Supabase: Handles configuration management and stores various configurations used across the platform.
- LangChain: Implements AI agents that handle narrative classification and geolocation mapping.
Each component interacts seamlessly, allowing the platform to process massive amounts of data while maintaining flexibility.
Data Pipeline & Processing
The data pipeline involves a series of orchestrated steps that allow the platform to mine, process, and enrich climate-related conversations. Here’s an overview of how data flows through the system:
- Data Ingestion: Media articles and social network posts are scraped from various sources.
- Data Transformation: AI agents process the raw data, classifying conversations and associating narratives.
- Enrichment: Posts are enriched with geolocation and narrative data, providing contextual insights into public discourse.
- Visualization: The enriched data is visualized, providing users with interactive maps and narrative analysis.
Use Cases
📊 Geographic Analysis of Climate Narratives
The platform classifies conversations into distinct discourses, including Biophysical, Critical, Dismissive, and Integrative, allowing users to explore climate narratives across regions.
This map shows the dominant climate change discourse in each U.S. state, based on social media conversations from March to May 2024.