# Metaflow An open-source framework for building, scaling, and managing real-life ML, AI, and data science projects. - Findings24: - ウェブサイト: - 料金: Free - カテゴリー: Developer Tools ## Metaflowとは？ Metaflow is an open-source Python framework for building and managing machine learning, artificial intelligence, and data science projects. It supports the full path from local exploration and debugging to production deployment, while tracking flow variables, data, and results automatically. Teams can run workflows on a laptop, scale compute in the cloud, and deploy to their own cloud account or an on-premise Kubernetes cluster. The framework is available under Apache License 2.0. You supply the Python code, libraries, data access, and execution environment. Local use and deployment to your own cloud or Kubernetes resources are different from purchasing a managed service; compute, storage, or external model services can have separate costs. ## Metaflowでできること ### Plain-Python workflows Author workflows in Python, develop and debug locally, and promote the same flow code to production after configuring the execution environment. ### Experiment and data tracking Metaflow stores variables inside flows and versions data as it moves between steps, supporting experiment analysis and debugging. ### Scalable cloud compute Run functions with GPUs, multiple cores, large amounts of memory, and multiple instances when local resources are insufficient. ### Production deployment and events Deploy flows with a single command and connect them to other systems through events, including workflows that react to changing data or real-time events. ### Flexible dependencies and data access Use Python libraries and manage dependencies locally or in the cloud, while accessing data from data warehouses and processing tabular data. ### Bring-your-own infrastructure Deploy the stack on AWS, Azure, Google Cloud, or a custom Kubernetes cluster. Configure data access, credentials, and governance for your environment; supporting your cloud is not a guarantee that every policy is automatically satisfied. ## 利用シーン ### Local model experimentation Explore in notebooks, develop flows incrementally, and test or debug on a laptop while automatically storing results for later analysis. ### Large-scale model training Move demanding training and generative AI workloads to cloud infrastructure with GPUs, multiple cores, or parallel instances. ### Production machine learning workflows Move a tested experimental flow into production using the same flow code, after setting up the relevant compute, storage, dependencies, and orchestrator. ### Event-driven data and AI systems Build workflows that respond automatically to updated data and other events, supporting reactive production systems. ### Team collaboration across infrastructure Organize shared experiments and results on a team’s cloud account or on-premise Kubernetes environment. Configure access and governance according to the team’s requirements. ## よくある質問 ### Who is Metaflow for? Metaflow is built for ML and AI engineers and data scientists working on machine learning, artificial intelligence, statistics, operations research, and related data projects. ### Can Metaflow be used locally before deployment? Yes. Explore with notebooks, develop and debug flows locally, and promote the same flow code to a configured production environment. The Python package is available for macOS and Linux. ### Which infrastructure does Metaflow support? Metaflow supports AWS with EKS and S3, AWS Batch, and AWS Step Functions; Azure with AKS and Azure Blob Storage; Google Cloud with GKE and Google Cloud Storage; and custom Kubernetes clusters. You configure the required infrastructure and access. ### Can Metaflow workflows react to events? Yes. Metaflow supports event integrations and workflows that can be triggered by updated data and other real-time events. ### Is Metaflow free to use? The framework is available under Apache License 2.0 without a software license fee. You remain responsible for license conditions and for the compute, storage, or external services you choose. A managed deployment or model service is separate from the open-source framework. ## クイック情報 - **Category**: Machine learning and data science framework - **License**: Apache License 2.0; no framework license fee - **Primary users**: ML and AI engineers, data scientists - **Development model**: Plain Python workflows developed locally and deployed to production - **Deployment environments**: AWS, Azure, Google Cloud, and custom Kubernetes clusters - **Local package**: Python package for macOS and Linux