diff --git a/flink/README.md b/flink/README.md new file mode 100644 index 000000000..efc6da199 --- /dev/null +++ b/flink/README.md @@ -0,0 +1,153 @@ + + +# Supported tags and respective `Dockerfile` links + +- [`1.2.1-hadoop2-scala_2.10` (*1.2/hadoop2-scala_2.10-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop2-scala_2.10-debian/Dockerfile) +- [`1.2.1-hadoop2-scala_2.11`, `1.2.1-hadoop2`, `1.2-hadoop2`, `hadoop2` (*1.2/hadoop2-scala_2.11-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop2-scala_2.11-debian/Dockerfile) +- [`1.2.1-hadoop24-scala_2.10` (*1.2/hadoop24-scala_2.10-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop24-scala_2.10-debian/Dockerfile) +- [`1.2.1-hadoop24-scala_2.11`, `1.2.1-hadoop24`, `1.2-hadoop24`, `hadoop24` (*1.2/hadoop24-scala_2.11-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop24-scala_2.11-debian/Dockerfile) +- [`1.2.1-hadoop26-scala_2.10` (*1.2/hadoop26-scala_2.10-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop26-scala_2.10-debian/Dockerfile) +- [`1.2.1-hadoop26-scala_2.11`, `1.2.1-hadoop26`, `1.2-hadoop26`, `hadoop26` (*1.2/hadoop26-scala_2.11-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop26-scala_2.11-debian/Dockerfile) +- [`1.2.1-hadoop27-scala_2.10`, `1.2.1-scala_2.10`, `1.2-scala_2.10`, `scala_2.10` (*1.2/hadoop27-scala_2.10-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop27-scala_2.10-debian/Dockerfile) +- [`1.2.1-hadoop27-scala_2.11`, `1.2.1-scala_2.11`, `1.2-scala_2.11`, `scala_2.11`, `1.2.1-hadoop27`, `1.2-hadoop27`, `hadoop27`, `1.2.1`, `1.2`, `latest` (*1.2/hadoop27-scala_2.11-debian/Dockerfile*)](https://github.com/docker-flink/docker-flink/blob/263dcb48ce49fcc129bbf09f331360549d573db5/1.2/hadoop27-scala_2.11-debian/Dockerfile) + +# Quick reference + +- **Where to get help**: + [Community & Project Info](https://flink.apache.org/community.html) + +- **Where to file issues**: + [https://github.com/docker-flink/docker-flink/issues](https://github.com/docker-flink/docker-flink/issues) + +- **Maintained by**: + [members of the Apache Flink community](https://github.com/docker-flink/docker-flink) + +- **Published image artifact details**: + [repo-info repo's `repos/flink/` directory](https://github.com/docker-library/repo-info/blob/master/repos/flink) ([history](https://github.com/docker-library/repo-info/commits/master/repos/flink)) + (image metadata, transfer size, etc) + +- **Image updates**: + [official-images PRs with label `library/flink`](https://github.com/docker-library/official-images/pulls?q=label%3Alibrary%2Fflink) + [official-images repo's `library/flink` file](https://github.com/docker-library/official-images/blob/master/library/flink) ([history](https://github.com/docker-library/official-images/commits/master/library/flink)) + +- **Source of this description**: + [docs repo's `flink/` directory](https://github.com/docker-library/docs/tree/master/flink) ([history](https://github.com/docker-library/docs/commits/master/flink)) + +- **Supported Docker versions**: + [the latest release](https://github.com/docker/docker/releases/latest) (down to 1.6 on a best-effort basis) + +# What is Apache Flink? + +Apache Flink is an open source stream processing framework with powerful stream- and batch-processing capabilities. + +Learn more about Flink at [https://flink.apache.org/](https://flink.apache.org/) + +> [wikipedia.org/wiki/Apache_Flink](https://en.wikipedia.org/wiki/Apache_Flink) + +![logo](https://raw.githubusercontent.com/docker-library/docs/71398f44551617e3934a86b4b7a3c770ae093b59/flink/logo.png) + +# How to use this Docker image + +## Run a Flink local cluster + +To run a single Flink local cluster: + +```console +$ docker run --name flink_local -p 8081:8081 -t flink local +``` + +Then with a web browser go to `http://localhost:8081/` to see the Flink Web Dashboard (adjust the hostname for your Docker host). + +To use Flink, you can submit a job to the cluster using the Web UI or you can also do it from a different Flink container, for example: + +```console +$ docker run --rm -t flink flink run -m -c +``` + +## Running a JobManager or a TaskManager + +You can run a JobManager (master). + +```console +$ docker run --name flink_jobmanager -d -t flink taskmanager +``` + +You can also run a TaskManager (worker). Notice that workers need to register with the JobManager directly or via ZooKeeper so the master starts to send them tasks to execute. + +```console +$ docker run --name flink_taskmanager -d -t flink taskmanager +``` + +## Running a cluster using Docker Compose + +With Docker Compose you can create a Flink cluster: + +```yml +version: "2.1" +services: + jobmanager: + image: ${FLINK_DOCKER_IMAGE_NAME:-flink} + expose: + - "6123" + ports: + - "8081:8081" + command: jobmanager + environment: + - JOB_MANAGER_RPC_ADDRESS=jobmanager + + taskmanager: + image: ${FLINK_DOCKER_IMAGE_NAME:-flink} + expose: + - "6121" + - "6122" + depends_on: + - jobmanager + command: taskmanager + links: + - "jobmanager:jobmanager" + environment: + - JOB_MANAGER_RPC_ADDRESS=jobmanager +``` + +and just run `docker-compose up`. + +Scale the cluster up or down to *N* TaskManagers: + +```console +docker-compose scale taskmanager= +``` + +## Configuration + +These are the default ports used by the Flink image: + +- The Web Client is on port `8081` +- JobManager RPC port `6123` +- TaskManagers RPC port `6122` +- TaskManagers Data port `6121` + +# About this repository + +This repository is available on [github.com/docker-flink/docker-flink](https://github.com/docker-flink/docker-flink), and the official build is on the [Docker Hub](https://hub.docker.com/_/flink/). + +This repository is maintained by members of the Apache Flink community. + +# License + +Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0