Add Flink 2.x to Factor House Local #13
Description
Flink 2.1.0 introduces new features such as ML model DDLs and unified real-time + AI support. It also provides native OpenLineage integration without job code changes, including full SQL and Table API coverage. Adding Flink 2 keeps Factor House Local aligned with the future of the Flink ecosystem.
We should add a Flink 2.1.0 environment alongside the existing Flink 1.x stack. This will include JobManager, TaskManager, and SQL Gateway services, integrated with Hive Metastore, PostgreSQL, and MinIO.
This enables experimentation with real-time ML inference directly in SQL, simplifies lineage tracking via native OpenLineage support, and allows developers to compare Flink 1.x and 2.x behavior side by side. It ensures Factor House Local is future-ready while maintaining backward compatibility.