zeus
4 years ago
37 changed files with 74824 additions and 0 deletions
|
After Width: | Height: | Size: 464 KiB |
|
After Width: | Height: | Size: 452 KiB |
|
After Width: | Height: | Size: 30 KiB |
|
After Width: | Height: | Size: 54 KiB |
|
After Width: | Height: | Size: 37 KiB |
|
After Width: | Height: | Size: 7.7 KiB |
@ -0,0 +1,556 @@ |
|||
= Swarm ! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
// Metadata: |
|||
:description: IoT Εισαγωγή στο Swarm |
|||
:keywords: Cloud, swarm |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Πίνακας περιεχομένων |
|||
:toclevels: 4 |
|||
:source-highlighter: coderay |
|||
//coderay, highlightjs, prettify |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
{empty} + |
|||
|
|||
== Swarm - Intro |
|||
|
|||
Docker Swarm is a *clustering* and *scheduling* tool for Docker containers. With Swarm, IT administrators and developers can establish and manage a cluster of Docker nodes as a single virtual system. |
|||
|
|||
Clustering is an important feature for container technology, because it creates a cooperative group of systems that can provide redundancy, enabling Docker Swarm failover if one or more nodes experience an outage. |
|||
|
|||
|
|||
Features: |
|||
|
|||
- Decentralized design: Instead of handling differentiation between node roles at deployment time, the Docker Engine handles any specialization at runtime. You can deploy both kinds of nodes, managers and workers, using the Docker Engine. |
|||
|
|||
|
|||
- Scaling: For each service, you can declare the number of tasks you want to run. When you scale up or down, the swarm manager automatically adapts by adding or removing tasks to maintain the desired state. |
|||
|
|||
- Desired state reconciliation: The swarm manager node constantly monitors the cluster state and reconciles any differences between the actual state and your expressed desired state. For example, if you set up a service to run 10 replicas of a container, and a worker machine hosting two of those replicas crashes, the manager creates two new replicas to replace the replicas that crashed. The swarm manager assigns the new replicas to workers that are running and available. |
|||
|
|||
- Multi-host networking: You can specify an overlay network for your services. The swarm manager automatically assigns addresses to the containers on the overlay network when it initializes or updates the application. |
|||
|
|||
- Service discovery: Swarm manager nodes assign each service in the swarm a unique DNS name and load balances running containers. You can query every container running in the swarm through a DNS server embedded in the swarm. |
|||
|
|||
- Load balancing: You can expose the ports for services to an external load balancer. Internally, the swarm lets you specify how to distribute service containers between nodes. |
|||
|
|||
- Secure by default: Each node in the swarm enforces TLS mutual authentication and encryption to secure communications between itself and all other nodes. You have the option to use self-signed root certificates or certificates from a custom root CA. |
|||
|
|||
- Rolling updates: At rollout time you can apply service updates to nodes incrementally. The swarm manager lets you control the delay between service deployment to different sets of nodes. If anything goes wrong, you can roll back to a previous version of the service. |
|||
|
|||
|
|||
|
|||
|
|||
== Swarm architecture |
|||
|
|||
image:./swarm-diagram.png[alt="Swarm"] + |
|||
|
|||
|
|||
There are two types of nodes: *managers* and *workers.* |
|||
|
|||
|
|||
=== Manager nodes |
|||
|
|||
Manager nodes handle cluster management tasks: |
|||
|
|||
- maintaining cluster state |
|||
- scheduling services |
|||
- serving swarm mode HTTP API endpoints |
|||
|
|||
Using a Raft implementation, the managers maintain a consistent internal state of the entire swarm and all the services running on it. |
|||
|
|||
To take advantage of swarm mode’s fault-tolerance features, Docker recommends you implement an odd number of nodes according to your organization’s high-availability requirements. When you have multiple managers you can recover from the failure of a manager node without downtime. |
|||
|
|||
- A three-manager swarm tolerates a maximum loss of one manager. |
|||
- A five-manager swarm tolerates a maximum simultaneous loss of two manager nodes. |
|||
- An N manager cluster tolerates the loss of at most (N-1)/2 managers. |
|||
|
|||
|
|||
=== Worker nodes |
|||
|
|||
Worker nodes are also instances of Docker Engine whose sole purpose is to execute containers. Worker nodes don’t participate in the Raft distributed state, make scheduling decisions, or serve the swarm mode HTTP API. |
|||
|
|||
You can create a swarm of one manager node, but you cannot have a worker node without at least one manager node. By default, all managers are also workers. |
|||
|
|||
|
|||
== Create swarm |
|||
|
|||
Initialize a swarm. The docker engine targeted by this command becomes a manager in the newly created single-node swarm. |
|||
|
|||
.create swarm |
|||
[source,sh] |
|||
---- |
|||
docker swarm init --advertise-addr ip |
|||
Swarm initialized: current node (bvz81updecsj6wjz393c09vti) is now a manager. |
|||
|
|||
To add a worker to this swarm, run the following command: |
|||
|
|||
docker swarm join \ |
|||
--token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-1awxwuwd3z9j1z3puu7rcgdbx \ |
|||
172.17.0.2:2377 |
|||
|
|||
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. |
|||
---- |
|||
|
|||
=== Join token |
|||
|
|||
Join tokens are secrets that allow a node to join the swarm. |
|||
|
|||
There are two different join tokens available, |
|||
|
|||
- one for the worker role and |
|||
- one for the manager role. |
|||
|
|||
|
|||
.join token |
|||
[source,sh] |
|||
---- |
|||
docker swarm join-token worker |
|||
To add a worker to this swarm, run the following command: |
|||
|
|||
docker swarm join \ |
|||
--token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-1awxwuwd3z9j1z3puu7rcgdbx \ |
|||
172.17.0.2:2377 |
|||
|
|||
docker swarm join-token manager |
|||
To add a manager to this swarm, run the following command: |
|||
|
|||
docker swarm join \ |
|||
--token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-7p73s1dx5in4tatdymyhg9hu2 \ |
|||
172.17.0.2:2377 |
|||
---- |
|||
|
|||
|
|||
=== Join swarm |
|||
|
|||
Join a node to a swarm. The node joins as a manager node or worker node based upon the token you pass with the --token flag. If you pass a manager token, the node joins as a manager. If you pass a worker token, the node joins as a worker. |
|||
|
|||
|
|||
.join worker |
|||
[source,sh] |
|||
---- |
|||
docker swarm join --token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-1awxwuwd3z9j1z3puu7rcgdbx --advertise-addr eth1:2377 192.168.99.121:2377 |
|||
This node joined a swarm as a worker. |
|||
---- |
|||
|
|||
.join manager |
|||
[source,sh] |
|||
---- |
|||
docker swarm join --token SWMTKN-1-3pu6hszjas19xyp7ghgosyx9k8atbfcr8p2is99znpy26u2lkl-7p73s1dx5in4tatdymyhg9hu2 --advertise-addr eth1:2377 192.168.99.121:2377 |
|||
This node joined a swarm as a manager. |
|||
---- |
|||
|
|||
=== Leave swarm |
|||
|
|||
When you run this command on a worker, that worker leaves the swarm. |
|||
|
|||
---- |
|||
docker swarm leave |
|||
---- |
|||
|
|||
You can use the *--force* option on a manager to remove it from the swarm. |
|||
|
|||
|
|||
[NOTE] |
|||
.INFO |
|||
==== |
|||
|
|||
The safe way to remove a manager from a swarm is to *demote* it to a worker and then direct it to leave the quorum without using --force |
|||
|
|||
---- |
|||
docker node demote id |
|||
---- |
|||
|
|||
|
|||
==== |
|||
|
|||
|
|||
|
|||
== Manage nodes |
|||
|
|||
|
|||
|
|||
[NOTE] |
|||
.INFO |
|||
==== |
|||
|
|||
This is a cluster management command, and must be executed on a swarm manager node. |
|||
|
|||
==== |
|||
|
|||
- *docker node ls* List nodes in the swarm |
|||
|
|||
---- |
|||
docker node ls |
|||
---- |
|||
|
|||
- *docker node demote* `_ Demote one or more nodes from manager in the swarm_` |
|||
- *docker node inspect* `_ Display detailed information on one or more nodes_` |
|||
- *docker node promote* `_ Promote one or more nodes to manager in the swarm_` |
|||
- *docker node ps* `_ List tasks running on one or more nodes, defaults to current node_` |
|||
- *docker node rm* `_ Remove one or more nodes from the swarm_` |
|||
- *docker node update* `_ Update a node_` |
|||
- *docker node inspect* `_ Display detailed information on one or more nodes_` |
|||
|
|||
---- |
|||
docker node inspect id # from ls |
|||
---- |
|||
|
|||
---- |
|||
[ |
|||
{ |
|||
"ID": "e216jshn25ckzbvmwlnh5jr3g", |
|||
"Version": { |
|||
"Index": 10 |
|||
}, |
|||
"CreatedAt": "2017-05-16T22:52:44.9910662Z", |
|||
"UpdatedAt": "2017-05-16T22:52:45.230878043Z", |
|||
"Spec": { |
|||
"Role": "manager", |
|||
"Availability": "active" |
|||
}, |
|||
"Description": { |
|||
"Hostname": "swarm-manager", |
|||
"Platform": { |
|||
"Architecture": "x86_64", |
|||
"OS": "linux" |
|||
}, |
|||
"Resources": { |
|||
"NanoCPUs": 1000000000, |
|||
"MemoryBytes": 1039843328 |
|||
}, |
|||
"Engine": { |
|||
"EngineVersion": "17.06.0-ce", |
|||
"Plugins": [ |
|||
{ |
|||
"Type": "Volume", |
|||
}, |
|||
"Status": { |
|||
"State": "ready", |
|||
"Addr": "168.0.32.137" |
|||
}, |
|||
"ManagerStatus": { |
|||
"Leader": true, |
|||
"Reachability": "reachable", |
|||
"Addr": "168.0.32.137:2377" |
|||
} |
|||
} |
|||
] |
|||
---- |
|||
|
|||
[NOTE] |
|||
.INFO |
|||
==== |
|||
|
|||
Filtering output with jq |
|||
|
|||
.jq |
|||
[source,sh] |
|||
---- |
|||
apt update |
|||
apt install jq |
|||
docker node inspect id | jq -r '.[]["Status"]["State"]' |
|||
---- |
|||
==== |
|||
|
|||
|
|||
|
|||
|
|||
== Deploy services and Tasks |
|||
|
|||
|
|||
|
|||
To deploy an application image when Docker Engine is in swarm mode, you *create a service.* |
|||
|
|||
A *service* is the image for a microservice within the context of some larger application. |
|||
|
|||
`_Examples of services might include an HTTP server, a database, or any other type of executable program that you wish to run in a distributed environment._` |
|||
|
|||
|
|||
|
|||
|
|||
A *task* is the atomic unit of scheduling within a swarm. |
|||
|
|||
When you declare a desired service state by creating or updating a service, the orchestrator realizes the desired state by scheduling tasks. |
|||
|
|||
`_For instance, you define a service that instructs the orchestrator to keep three instances of an HTTP listener running at all times._` |
|||
|
|||
The orchestrator responds by creating three tasks. |
|||
|
|||
Each task is a slot that the scheduler fills by spawning a container. |
|||
|
|||
image:./services-diagram.png[alt="services"] + |
|||
|
|||
|
|||
=== service vs stack |
|||
|
|||
- A *Service* defines one or more instances of a single image deployed on one or more machines (described by one entry in the services part of *yaml* files). |
|||
|
|||
- A *Stack* defines a group of heterogeneous services (described by the whole yaml file). |
|||
|
|||
|
|||
The *docker service* command is used when managing individual service on a docker swarm cluster. |
|||
|
|||
|
|||
The *docker stack* command can be used to manage a multi-service application. |
|||
|
|||
|
|||
=== Build |
|||
|
|||
[NOTE] |
|||
.INFO |
|||
==== |
|||
|
|||
This is a cluster management command, and must be executed on a swarm manager node. |
|||
|
|||
==== |
|||
|
|||
==== Create yaml |
|||
|
|||
.yml example MPI (save as run.yml) |
|||
[source, yaml] |
|||
---- |
|||
version: "3.8" |
|||
|
|||
services: |
|||
|
|||
master: |
|||
image: image |
|||
user: root |
|||
environment: |
|||
# Pass environment variables to containers CUSTOM |
|||
- PASSWORD=dgergergergerrfgwehrtsger |
|||
- PASSWORDVIEW=rtyrwtyrwftertgueteyserfy5e6ytrg |
|||
- SERVERROLE=master |
|||
- SERVERWEB=no |
|||
# Pass environment variables to containers FROM ENGINE see inspect |
|||
- NODENAME={{.Node.Hostname}} |
|||
- NODEID={{.Node.ID}} |
|||
- SERVICEID={{.Service.ID}} |
|||
- SERVICENAME={{.Service.Name}} |
|||
- TASKID={{.Task.ID}} |
|||
- TASKNAME={{.Task.Name}} |
|||
- TASKREPID={{.Task.Slot}} |
|||
deploy: |
|||
# mode: global # see image replica-vs-global |
|||
replicas: 1 |
|||
placement: |
|||
max_replicas_per_node: 1 |
|||
constraints: |
|||
- node.role == worker |
|||
#- node.labels.region == region1 |
|||
resources: |
|||
limits: |
|||
cpus: '0.50' |
|||
memory: 500M |
|||
reservations: |
|||
cpus: '0.25' |
|||
memory: 200M |
|||
restart_policy: |
|||
condition: on-failure |
|||
delay: 5s |
|||
max_attempts: 5 |
|||
window: 120s |
|||
update_config: |
|||
parallelism: 2 |
|||
delay: 10s |
|||
order: stop-first |
|||
networks: |
|||
mpi2-net: |
|||
ports: |
|||
- "5580:80" |
|||
- "5588:8088" |
|||
|
|||
slave: |
|||
image: image |
|||
user: root |
|||
environment: |
|||
- PASSWORD=rtyrthrthyrthyrtyrtyrty |
|||
- PASSWORDVIEW=rtyrtuyrtuyrt |
|||
- SERVERROLE=slave |
|||
- SERVERWEB=no |
|||
- NODENAME={{.Node.Hostname}} |
|||
- NODEID={{.Node.ID}} |
|||
- SERVICEID={{.Service.ID}} |
|||
- SERVICENAME={{.Service.Name}} |
|||
- TASKID={{.Task.ID}} |
|||
- TASKNAME={{.Task.Name}} |
|||
- TASKREPID={{.Task.Slot}} |
|||
deploy: |
|||
# mode: global # see image replica-vs-global |
|||
replicas: 9 |
|||
placement: |
|||
max_replicas_per_node: 1 |
|||
constraints: |
|||
- node.role == worker |
|||
- node.labels.region == region2 |
|||
resources: |
|||
limits: |
|||
cpus: '0.50' |
|||
memory: 500M |
|||
reservations: |
|||
cpus: '0.25' |
|||
memory: 200M |
|||
restart_policy: |
|||
condition: on-failure |
|||
delay: 5s |
|||
max_attempts: 5 |
|||
window: 120s |
|||
update_config: |
|||
parallelism: 2 |
|||
delay: 10s |
|||
order: stop-first |
|||
networks: |
|||
mpi2-net: |
|||
ports: |
|||
- "5581:80" |
|||
- "5590:8088" |
|||
|
|||
|
|||
web: |
|||
image: image |
|||
user: root |
|||
environment: |
|||
- SERVERROLE=web |
|||
- SERVERWEB=yes |
|||
- NODENAME={{.Node.Hostname}} |
|||
- NODEID={{.Node.ID}} |
|||
- SERVICEID={{.Service.ID}} |
|||
- SERVICENAME={{.Service.Name}} |
|||
- TASKID={{.Task.ID}} |
|||
- TASKNAME={{.Task.Name}} |
|||
- TASKREPID={{.Task.Slot}} |
|||
deploy: |
|||
# mode: global # see image replica-vs-global |
|||
replicas: 1 |
|||
placement: |
|||
constraints: |
|||
- node.role == worker |
|||
- node.labels.region == region3 |
|||
resources: |
|||
limits: |
|||
cpus: '0.50' |
|||
memory: 500M |
|||
reservations: |
|||
cpus: '0.25' |
|||
memory: 200M |
|||
restart_policy: |
|||
condition: on-failure |
|||
delay: 5s |
|||
max_attempts: 5 |
|||
window: 120s |
|||
update_config: |
|||
parallelism: 2 |
|||
delay: 10s |
|||
order: stop-first |
|||
networks: |
|||
mpi2-net: |
|||
ports: |
|||
- "5598:80" |
|||
- "5599:8088" |
|||
|
|||
|
|||
networks: |
|||
mpi2-net: |
|||
|
|||
|
|||
|
|||
---- |
|||
|
|||
.replicas-vs-global |
|||
image:./replicated-vs-global.png[alt="replica-vs-global"] + |
|||
|
|||
|
|||
|
|||
[NOTE] |
|||
.INFO |
|||
==== |
|||
YAML (a recursive acronym for "YAML Ain't Markup Language") is a human-readable data-serialization language. |
|||
|
|||
It is commonly used for configuration files and in applications where data is being stored or transmitted. |
|||
|
|||
YAML targets many of the same communications applications as Extensible Markup Language (XML) but has a minimal syntax which intentionally differs from SGML |
|||
|
|||
It uses Python-style indentation to indicate nesting |
|||
|
|||
From Wikipedia, the free encyclopedia + |
|||
|
|||
more: Learn YAML in five minutes! |
|||
https://www.codeproject.com/Articles/1214409/Learn-YAML-in-five-minutes[^] |
|||
|
|||
==== |
|||
|
|||
|
|||
==== Build |
|||
|
|||
|
|||
.build |
|||
[source,sh] |
|||
---- |
|||
docker stack deploy -c run.yml my_service |
|||
---- |
|||
|
|||
|
|||
==== List services |
|||
|
|||
.List the services that are running as part of the specified stack. |
|||
[source,sh] |
|||
---- |
|||
docker stack services |
|||
---- |
|||
|
|||
|
|||
|
|||
.List *ALL* services are running in the swarm. |
|||
[source,sh] |
|||
---- |
|||
docker service ls |
|||
---- |
|||
|
|||
|
|||
|
|||
|
|||
==== Remove one or more stacks |
|||
|
|||
.Remove the stack from the swarm. |
|||
[source,sh] |
|||
---- |
|||
docker stack rm |
|||
---- |
|||
|
|||
.Removes the specified services from the swarm. |
|||
[source,sh] |
|||
---- |
|||
docker service rm |
|||
---- |
|||
|
|||
|
|||
==== List tasks |
|||
|
|||
.List the tasks that are running as part of the specified services. |
|||
[source,sh] |
|||
---- |
|||
docker service ps |
|||
---- |
|||
|
|||
|
|||
.List the tasks in the stack |
|||
[source,sh] |
|||
---- |
|||
docker stack ps |
|||
---- |
|||
|
|||
|
|||
[NOTE] |
|||
.INFO |
|||
**** |
|||
Command-line completion (also tab completion) is a common feature of command-line interpreters, in which the program automatically fills in partially typed commands. |
|||
|
|||
more info: https://en.wikipedia.org/wiki/Command-line_completion[^] |
|||
**** |
|||
|
|||
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
@ -0,0 +1,460 @@ |
|||
= Cloud ! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
// Metadata: |
|||
:description: IoT Εισαγωγή στο Cloud |
|||
:keywords: Cloud, swarm |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Πίνακας περιεχομένων |
|||
:toclevels: 4 |
|||
:source-highlighter: highlight |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
{empty} + |
|||
|
|||
== Cloud - Intro |
|||
|
|||
Cloud computing is the **on-demand availability of computer system resources**, especially data storage and computing power, without direct active management by the user. The term is generally used to describe data centers available to many users over the Internet. |
|||
|
|||
Large clouds, predominant today, often have functions distributed over multiple locations from central servers. If the connection to the user is relatively close, it may be designated an edge server. |
|||
|
|||
Clouds may be limited to a single organization (enterprise clouds), or be available to many organizations (public cloud). |
|||
|
|||
Cloud computing relies on sharing of resources to achieve coherence and economies of scale. |
|||
|
|||
image:./Cloud_computing.svg.png[alt="Cloud_computing"] + |
|||
From Wikipedia, the free encyclopedia + |
|||
https://en.wikipedia.org/wiki/Cloud_computing[^] |
|||
|
|||
|
|||
|
|||
|
|||
=== Cloud Computing Tutorial for Beginners |
|||
|
|||
* Cloud Computing Tutorial for Beginners |
|||
+ |
|||
video::RWgW-CgdIk0[youtube] |
|||
|
|||
|
|||
== Cloud computing architecture |
|||
|
|||
Cloud computing architecture refers to the components and subcomponents required for cloud computing. These components typically consist of a front end platform (fat client, thin client, mobile device), back end platforms (servers, storage), a cloud based delivery, and a network (Internet, Intranet, Intercloud). Combined, these components make up cloud computing architecture. |
|||
|
|||
=== Virtualization |
|||
|
|||
In computing, virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, storage devices, and computer network resources. |
|||
|
|||
=== Containerization |
|||
|
|||
Containerization has become a major trend in software development as an alternative or companion to virtualization. It involves encapsulating or packaging up software code and all its dependencies so that it can run uniformly and consistently on any infrastructure. The technology is quickly maturing, resulting in measurable benefits for developers and operations teams as well as overall software infrastructure. |
|||
|
|||
=== Virtual Machines vs Docker Containers |
|||
|
|||
- A container image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it. |
|||
|
|||
Docker is the service to run multiple containers on a machine (node) which can be on a vitual machine or on a physical machine. |
|||
|
|||
- A virtual machine is an entire operating system (which normally is not lightweight). |
|||
|
|||
|
|||
* Virtual Machines vs Docker Containers |
|||
+ |
|||
video::TvnZTi_gaNc[youtube] |
|||
|
|||
==== Docker Containers |
|||
|
|||
image:./container-what-is-container.png[alt="Container",align="center",width=550,height=550] |
|||
|
|||
|
|||
==== Virtual Machines |
|||
|
|||
image:./container-vm-whatcontainer_2.png[alt="VirtualMachine",align="center",width=550,height=550] |
|||
|
|||
== Orchestration |
|||
|
|||
Container orchestration automates the deployment, management, scaling, and networking of containers. Enterprises that need to deploy and manage hundreds or thousands of Linux® containers and hosts can benefit from container orchestration. |
|||
Container orchestration can be used in any environment where you use containers. It can help you to deploy the same application across different environments without needing to redesign it. And microservices in containers make it easier to orchestrate services, including storage, networking, and security. |
|||
|
|||
Containers give your microservice-based apps an ideal application deployment unit and self-contained execution environment. They make it possible to run multiple parts of an app independently in microservices, on the same hardware, with much greater control over individual pieces and life cycles. |
|||
|
|||
Managing the lifecycle of containers with orchestration also supports DevOps teams who integrate it into CI/CD workflows. Along with application programming interfaces (APIs) and DevOps teams, containerized microservices are the foundation for cloud-native applications. |
|||
|
|||
|
|||
Container orchestration used for: |
|||
|
|||
- Provisioning and deployment |
|||
- Configuration and scheduling |
|||
- Resource allocation |
|||
- Container availability |
|||
- Scaling or removing containers based on balancing workloads across your infrastructure |
|||
- Load balancing and traffic routing |
|||
- Monitoring container health |
|||
- Configuring applications based on the container in which they will run |
|||
- Keeping interactions between containers secure |
|||
|
|||
=== Kubernetes vs Docker Swarm |
|||
|
|||
* Kubernetes vs Docker Swarm |
|||
+ |
|||
video::FmrAGliHvzQ[youtube] |
|||
|
|||
=== Technical Comparisons |
|||
|
|||
image:./1-2-683x1024.png[alt="dockerSwarmVsKubernetes"] + |
|||
image:./2-1-683x1024.png[alt="dockerSwarmVsKubernetes"] + |
|||
|
|||
|
|||
|
|||
=== Conclusion |
|||
|
|||
When comparing Docker Swarm vs Kubernetes, it becomes apparent that the origins of both platforms have played key roles in shaping their features and communities today. |
|||
|
|||
Docker, realizing the strength of its container technology, decided to build a platform that made it simple for Docker users to begin orchestrating their container workloads across multiple nodes. However, their desire to preserve this tight coupling can be said to have limited the extensibility of the platform. |
|||
|
|||
Kubernetes, on the other hand, took key concepts taken from Google Borg, and, from a high level perspective, decided to make containerization fit into the former platform’s existing workload orchestration model. This resulted in Kubernetes emphasis on reliability, sometimes at the cost of simplicity and performance. |
|||
|
|||
|
|||
=== Popularity of searches for each platform |
|||
|
|||
image:./Screen-Shot-2018-01-09-at-5.59.05-PM-700x410.png[alt="dockerSwarmVsKubernetes"] + |
|||
|
|||
Origin: |
|||
https://www.nirmata.com/2018/01/15/orchestration-platforms-in-the-ring-kubernetes-vs-docker-swarm |
|||
|
|||
=== Short_answer |
|||
|
|||
.Info |
|||
[NOTE] |
|||
==== |
|||
|
|||
* Docker:!footnote:disclaimer[https://stackoverflow.com/questions/47536536/whats-the-difference-between-docker-compose-and-kubernetes[origin info]] |
|||
|
|||
** Docker is the container technology that allows you to *containerize your applications.* |
|||
** Docker is *the core of using other technologies.* |
|||
|
|||
* Docker Compose |
|||
|
|||
** Docker Compose allows configuring and starting *multiple Docker containers.* |
|||
** Docker Compose is mostly used as a helper when you want to start multiple Docker containers and doesn't want to start each one separately using docker run .... |
|||
** Docker Compose is used for *starting containers on the SAME host.* |
|||
** Docker Compose is used instead of all optional parameters when building and running a single docker container. |
|||
|
|||
* Docker Swarm |
|||
|
|||
** Docker swarm is for *running* and connecting containers on *multiple hosts.* |
|||
** Docker swarm is a *container cluster management and orchestration tool.* |
|||
*** It manages containers running on multiple hosts and does things like scaling, starting a new container when one crashes, networking containers ... |
|||
** Docker swarm is docker in *production.* |
|||
** It is the *native docker orchestration tool* that is embedded in the Docker Engine. |
|||
** The docker swarm file named stack file is very similar to a Docker compose file. |
|||
|
|||
* Kubernetes |
|||
|
|||
** Kubernetes is a *container orchestration tool* developed by Google. |
|||
** Kubernete's goal is *very similar* to that for Docker swarm. |
|||
|
|||
==== |
|||
|
|||
.Update |
|||
[NOTE] |
|||
==== |
|||
Docker support https://github.com/docker/compose-on-kubernetes[docker stack deploy --orchestrator=kubernetes] https://docs.docker.com/engine/reference/commandline/stack_deploy/#options[options] |
|||
|
|||
==== |
|||
|
|||
|
|||
== Docker |
|||
|
|||
|
|||
=== Images |
|||
|
|||
In Docker, everything is based on Images. An image is a combination of a file system and parameters. |
|||
|
|||
==== Dockerfile |
|||
|
|||
A Dockerfile is a simple text file that contains a list of commands that the Docker client calls while creating an image. It's a simple way to automate the image creation process. The best part is that the commands you write in a Dockerfile are almost identical to their equivalent Linux commands. This means you don't really have to learn new syntax to create your own dockerfiles. |
|||
|
|||
.Dockerfile |
|||
[source,sh] |
|||
---- |
|||
FROM ubuntu:16.04 |
|||
ENV DEBIAN_FRONTEND noninteractive |
|||
RUN apt-get update -y && \ |
|||
apt-get -y install gcc && \ |
|||
rm -rf /var/lib/apt/lists/* |
|||
---- |
|||
|
|||
==== docker build |
|||
|
|||
.docker build |
|||
[source,sh] |
|||
---- |
|||
docker build -t ImageName:TagName dir |
|||
|
|||
Options |
|||
|
|||
-t − is to mention a tag to the image |
|||
|
|||
ImageName − This is the name you want to give to your image. |
|||
|
|||
TagName − This is the tag you want to give to your image. |
|||
|
|||
Dir − The directory where the Docker File is present. |
|||
---- |
|||
|
|||
.docker build example |
|||
[source,sh] |
|||
---- |
|||
docker build –t myimage:0.1 . |
|||
---- |
|||
|
|||
|
|||
==== Displaying Docker Images |
|||
|
|||
To see the list of Docker images on the system, you can issue the following command. |
|||
|
|||
.docker images |
|||
[source,sh] |
|||
---- |
|||
docker images |
|||
---- |
|||
|
|||
This command is used to display all the images currently installed on the system. |
|||
|
|||
|
|||
**Output:** |
|||
|
|||
- TAG − This is used to logically tag images. |
|||
- Image ID − This is used to uniquely identify the image. |
|||
- Created − The number of days since the image was created. |
|||
- Virtual Size − The size of the image. |
|||
|
|||
|
|||
|
|||
|
|||
==== Removing Docker Images |
|||
|
|||
The Docker images on the system can be removed via the docker rmi command. |
|||
|
|||
.docker images |
|||
[source,sh] |
|||
---- |
|||
docker rmi |
|||
|
|||
This command is used to remove Docker images. |
|||
Syntax |
|||
|
|||
docker rmi ImageID |
|||
---- |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
==== Docker Hub |
|||
|
|||
Docker Hub is a registry service on the cloud that allows you to download Docker images that are built by other communities. You can also upload your own Docker built images to Docker hub. |
|||
|
|||
To run apache, you need to run the following command: |
|||
|
|||
.run docker image from Docker Hub |
|||
[source,sh] |
|||
---- |
|||
docker run -p 8080:80 apache |
|||
|
|||
Note the following points about the above command − |
|||
|
|||
|
|||
Here, apache is the name of the image we want to download from Docker hub and install on our Ubuntu machine. |
|||
|
|||
-p is used to map the port number of the internal Docker image to our main Ubuntu server so that we can access the container accordingly. |
|||
---- |
|||
|
|||
|
|||
|
|||
=== Containers |
|||
|
|||
Containers are instances of Docker images that can be run using the Docker run command. The basic purpose of Docker is to run containers. |
|||
|
|||
|
|||
==== Running a Container |
|||
|
|||
Running of containers is managed with the Docker run command. To run a container in an interactive mode, first launch the Docker container. |
|||
|
|||
.run docker image |
|||
[source,sh] |
|||
---- |
|||
docker run –it myimage /bin/bash |
|||
---- |
|||
|
|||
|
|||
==== Listing of Containers |
|||
|
|||
One can list all of the containers on the machine via the docker ps command. This command is used to return the currently running containers. |
|||
|
|||
.run docker image |
|||
[source,sh] |
|||
---- |
|||
docker ps |
|||
---- |
|||
|
|||
==== Display the running processes of a container |
|||
|
|||
|
|||
With this command, you can see the top processes within a container. |
|||
Syntax |
|||
|
|||
.docker top |
|||
[source,sh] |
|||
---- |
|||
docker top ContainerID |
|||
|
|||
Options |
|||
|
|||
ContainerID − This is the Container ID for which you want to see the top processes. |
|||
---- |
|||
|
|||
==== Stop a running container |
|||
|
|||
This command is used to stop a running container. |
|||
|
|||
.docker stop |
|||
[source,sh] |
|||
---- |
|||
docker stop ContainerID |
|||
|
|||
Options |
|||
|
|||
ContainerID − This is the Container ID which needs to be stopped. |
|||
---- |
|||
|
|||
==== Attach a running container |
|||
|
|||
|
|||
This command is used to attach to a running container. |
|||
|
|||
.docker |
|||
[source,sh] |
|||
---- |
|||
docker attach ContainerID |
|||
|
|||
Options |
|||
|
|||
ContainerID − This is the Container ID to which you need to attach. |
|||
---- |
|||
|
|||
|
|||
==== Delete container |
|||
|
|||
This command is used to delete a container. |
|||
|
|||
.docker rm |
|||
[source,sh] |
|||
---- |
|||
docker rm ContainerID |
|||
|
|||
Options |
|||
|
|||
ContainerID − This is the Container ID which needs to be removed. |
|||
---- |
|||
|
|||
==== Container Logging |
|||
|
|||
Logging is also available at the container level. |
|||
|
|||
.docker log |
|||
[source,sh] |
|||
---- |
|||
Docker logs containerID |
|||
|
|||
Parameters |
|||
|
|||
containerID − This is the ID of the container for which you need to see the logs. |
|||
---- |
|||
|
|||
=== Volumes |
|||
|
|||
Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. |
|||
|
|||
.docker volumes |
|||
[source,sh] |
|||
---- |
|||
docker run -d --name mycontainer -v /var/www/html:/var/html nginx:latest |
|||
---- |
|||
|
|||
|
|||
=== repositories |
|||
|
|||
You might have the need to have your own private repositories. You may not want to host the repositories on Docker Hub. For this, there is a repository container itself from Docker. Let’s see how we can download and use the container for registry. |
|||
|
|||
|
|||
==== Create |
|||
|
|||
.docker registry |
|||
[source,sh] |
|||
---- |
|||
docker run –d –p 5000:5000 –-name registry registry:2 |
|||
|
|||
The following points need to be noted about the above command: |
|||
|
|||
Registry is the container managed by Docker which can be used to host private repositories. |
|||
|
|||
The port number exposed by the container is 5000. Hence with the –p command, we are mapping the same port number to the 5000 port number on our localhost. |
|||
|
|||
We are just tagging the registry container as “2”, to differentiate it on the Docker host. |
|||
|
|||
The –d option is used to run the container in detached mode. This is so that the container can run in the background |
|||
---- |
|||
|
|||
|
|||
==== Push |
|||
|
|||
use the Docker push command to push the image to our private repository. |
|||
|
|||
.docker registry |
|||
[source,sh] |
|||
---- |
|||
docker push localhost:5000/myimage |
|||
---- |
|||
|
|||
==== Pull |
|||
|
|||
use the following Docker pull command to pull image from our private repository. |
|||
|
|||
.docker registry |
|||
[source,sh] |
|||
---- |
|||
docker pull localhost:5000/myimage |
|||
---- |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
:hardbreaks: |
|||
|
|||
{empty} + |
|||
{empty} + |
|||
{empty} |
|||
|
|||
:!hardbreaks: |
|||
|
|||
''' |
|||
|
|||
.Reminder |
|||
[NOTE] |
|||
==== |
|||
:hardbreaks: |
|||
Caminante, no hay camino, |
|||
se hace camino al andar. |
|||
|
|||
Wanderer, there is no path, |
|||
the path is made by walking. |
|||
|
|||
*Antonio Machado* Campos de Castilla |
|||
==== |
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 68 KiB |
|
After Width: | Height: | Size: 185 KiB |
|
After Width: | Height: | Size: 119 KiB |
|
After Width: | Height: | Size: 293 KiB |
|
After Width: | Height: | Size: 106 KiB |
|
After Width: | Height: | Size: 50 KiB |
@ -0,0 +1,205 @@ |
|||
= Swarm deploy Yaml ! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
:description: IoT Εισαγωγή στο Cloud |
|||
:keywords: Cloud, swarm |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Πίνακας περιεχομένων |
|||
:toclevels: 4 |
|||
:source-highlighter: coderay |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
{empty} + |
|||
|
|||
== Create Swarm Service YAML |
|||
|
|||
.YAML |
|||
[source,yaml] |
|||
---- |
|||
version: "3.4" |
|||
|
|||
services: |
|||
|
|||
# όνομα υπηρεσίας |
|||
master: |
|||
image: registry.vlabs.uniwa.gr:5080/swarmlab-service-mpi2 |
|||
user: root |
|||
# ENTRYPOINT instruction allows you to configure a container that will run as an executable. |
|||
entrypoint: ["mpi_bootstrap", "role=master", "mpi_master_service_name=master", "mpi_worker_service_name=worker"] |
|||
# Environment variables (declared with the ENV statement) can also be used in certain instructions as variables to be interpreted by the Dockerfile. |
|||
# https://docs.docker.com/engine/reference/builder/#environment-replacement |
|||
environment: |
|||
- PASSWORD=padatest |
|||
- PASSWORDVIEW=padatestview |
|||
- SERVERROLE=master |
|||
- SERVERWEB=no |
|||
# docker service inspect ondemand_mpi2_master |
|||
- NODENAME={{.Node.Hostname}} |
|||
- NODEID={{.Node.ID}} |
|||
- SERVICEID={{.Service.ID}} |
|||
- SERVICENAME={{.Service.Name}} |
|||
- TASKID={{.Task.ID}} |
|||
- TASKNAME={{.Task.Name}} |
|||
- TASKREPID={{.Task.Slot}} |
|||
# Specify configuration related to the deployment and running of services. |
|||
deploy: |
|||
# If the service is replicated (which is the default), specify the number of containers that should be running at any given time. |
|||
# In global mode, running one replica of service per swarm node. The number of global replicas is equal to the number of swarm nodes. In replica mode, you can run any number of service instances. |
|||
replicas: 9 |
|||
placement: |
|||
# max_replicas_per_node: 1 |
|||
constraints: |
|||
- node.role == worker |
|||
# Configures resource constraints. |
|||
# resources: |
|||
# limits: |
|||
# cpus: '0.50' |
|||
# memory: 500M |
|||
# reservations: |
|||
# cpus: '0.25' |
|||
|
|||
|
|||
|
|||
|
|||
# memory: 200M |
|||
# Configures how the service should be rollbacked in case of a failing update. |
|||
# |
|||
# parallelism: The number of containers to rollback at a time. If set to 0, all containers rollback simultaneously. |
|||
# delay: The time to wait between each container group’s rollback (default 0s). |
|||
# failure_action: What to do if a rollback fails. One of continue or pause (default pause) |
|||
# monitor: Duration after each task update to monitor for failure (ns|us|ms|s|m|h) (default 0s). |
|||
# max_failure_ratio: Failure rate to tolerate during a rollback (default 0). |
|||
# order: Order of operations during rollbacks. One of stop-first (old task is stopped before starting new one), or start-first (new task is started first, and the running tasks briefly overlap) (default stop-first). |
|||
# |
|||
restart_policy: |
|||
condition: on-failure |
|||
delay: 5s |
|||
max_attempts: 5 |
|||
window: 120s |
|||
|
|||
#Configures how the service should be updated. Useful for configuring rolling updates. |
|||
# |
|||
# parallelism: The number of containers to update at a time. |
|||
# delay: The time to wait between updating a group of containers. |
|||
# failure_action: What to do if an update fails. One of continue, rollback, or pause (default: pause). |
|||
# monitor: Duration after each task update to monitor for failure (ns|us|ms|s|m|h) (default 0s). |
|||
# max_failure_ratio: Failure rate to tolerate during an update. |
|||
# order: Order of operations during updates. One of stop-first (old task is stopped before starting new one), or start-first (new task is started first, and the running tasks briefly overlap) (default stop-first) Note: Only supported for v3.4 and higher. |
|||
# |
|||
# |
|||
update_config: |
|||
parallelism: 2 |
|||
delay: 10s |
|||
order: stop-first |
|||
networks: |
|||
mpi2-net: |
|||
volumes: |
|||
- mpi3_vol:/var/share |
|||
ports: |
|||
- "55520:80" |
|||
- "55521:8088" |
|||
- "55522:6088" |
|||
- "55523:6080" |
|||
|
|||
|
|||
worker: |
|||
image: registry.vlabs.uniwa.gr:5080/swarmlab-service-mpi2 |
|||
user: root |
|||
entrypoint: ["mpi_bootstrap", "role=worker", "mpi_master_service_name=master", "mpi_worker_service_name=worker"] |
|||
environment: |
|||
- SERVERROLE=worker |
|||
- SERVERWEB=no |
|||
- NODENAME={{.Node.Hostname}} |
|||
- NODEID={{.Node.ID}} |
|||
- SERVICEID={{.Service.ID}} |
|||
- SERVICENAME={{.Service.Name}} |
|||
- TASKID={{.Task.ID}} |
|||
- TASKNAME={{.Task.Name}} |
|||
- TASKREPID={{.Task.Slot}} |
|||
deploy: |
|||
replicas: 5 |
|||
placement: |
|||
# max_replicas_per_node: 1 |
|||
constraints: |
|||
- node.role == worker |
|||
#- node.id == ${worker} |
|||
# resources: |
|||
# limits: |
|||
# cpus: '0.50' |
|||
# memory: 500M |
|||
# reservations: |
|||
# cpus: '0.25' |
|||
# memory: 200M |
|||
restart_policy: |
|||
condition: on-failure |
|||
delay: 5s |
|||
max_attempts: 5 |
|||
window: 120s |
|||
update_config: |
|||
parallelism: 2 |
|||
delay: 10s |
|||
order: stop-first |
|||
networks: |
|||
mpi2-net: |
|||
volumes: |
|||
- mpi3_vol:/var/share |
|||
|
|||
web: |
|||
image: registry.vlabs.uniwa.gr:5080/swarmlab-service-sshfs |
|||
user: root |
|||
entrypoint: ["mpi_bootstrap", "role=worker", "mpi_master_service_name=master", "mpi_worker_service_name=worker"] |
|||
environment: |
|||
- SERVERROLE=worker |
|||
- SERVERWEB=yes |
|||
- NODENAME={{.Node.Hostname}} |
|||
- NODEID={{.Node.ID}} |
|||
- SERVICEID={{.Service.ID}} |
|||
- SERVICENAME={{.Service.Name}} |
|||
- TASKID={{.Task.ID}} |
|||
- TASKNAME={{.Task.Name}} |
|||
- TASKREPID={{.Task.Slot}} |
|||
deploy: |
|||
replicas: 1 |
|||
placement: |
|||
constraints: |
|||
- node.role == worker |
|||
resources: |
|||
limits: |
|||
cpus: '0.50' |
|||
memory: 500M |
|||
reservations: |
|||
cpus: '0.25' |
|||
memory: 200M |
|||
restart_policy: |
|||
condition: on-failure |
|||
delay: 5s |
|||
max_attempts: 5 |
|||
window: 120s |
|||
update_config: |
|||
parallelism: 2 |
|||
delay: 10s |
|||
order: stop-first |
|||
networks: |
|||
mpi2-net: |
|||
ports: |
|||
- "55519:80" |
|||
|
|||
#Creates a new network. The DRIVER accepts bridge or overlay which are the built-in network drivers. |
|||
#Bridge networks are isolated networks on a single Engine installation. If you want to create a network that spans multiple Docker hosts each running an Engine, you must create an overlay network. Unlike bridge networks, overlay networks require some pre-existing conditions before you can create one. |
|||
# |
|||
networks: |
|||
mpi2-net: |
|||
|
|||
|
|||
#Mount host paths or named volumes |
|||
#Creates a new volume that containers can consume and store data in. |
|||
volumes: |
|||
mpi3_vol: |
|||
external: false |
|||
|
|||
---- |
|||
|
|||
|
|||
@ -0,0 +1,750 @@ |
|||
<!DOCTYPE html> |
|||
<html lang="en"> |
|||
<head> |
|||
<meta charset="UTF-8"> |
|||
<meta http-equiv="X-UA-Compatible" content="IE=edge"> |
|||
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
|||
<meta name="generator" content="Asciidoctor 2.0.10"> |
|||
<meta name="description" content="IoT Εισαγωγή στο Cloud"> |
|||
<meta name="keywords" content="Cloud, swarm"> |
|||
<meta name="author" content="Apostolos rootApostolos@swarmlab.io"> |
|||
<title>Swarm deploy Yaml !</title> |
|||
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"> |
|||
<style> |
|||
/* Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */ |
|||
/* Uncomment @import statement to use as custom stylesheet */ |
|||
/*@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700";*/ |
|||
article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section{display:block} |
|||
audio,video{display:inline-block} |
|||
audio:not([controls]){display:none;height:0} |
|||
html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%} |
|||
a{background:none} |
|||
a:focus{outline:thin dotted} |
|||
a:active,a:hover{outline:0} |
|||
h1{font-size:2em;margin:.67em 0} |
|||
abbr[title]{border-bottom:1px dotted} |
|||
b,strong{font-weight:bold} |
|||
dfn{font-style:italic} |
|||
hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0} |
|||
mark{background:#ff0;color:#000} |
|||
code,kbd,pre,samp{font-family:monospace;font-size:1em} |
|||
pre{white-space:pre-wrap} |
|||
q{quotes:"\201C" "\201D" "\2018" "\2019"} |
|||
small{font-size:80%} |
|||
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline} |
|||
sup{top:-.5em} |
|||
sub{bottom:-.25em} |
|||
img{border:0} |
|||
svg:not(:root){overflow:hidden} |
|||
figure{margin:0} |
|||
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em} |
|||
legend{border:0;padding:0} |
|||
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0} |
|||
button,input{line-height:normal} |
|||
button,select{text-transform:none} |
|||
button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer} |
|||
button[disabled],html input[disabled]{cursor:default} |
|||
input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0} |
|||
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0} |
|||
textarea{overflow:auto;vertical-align:top} |
|||
table{border-collapse:collapse;border-spacing:0} |
|||
*,*::before,*::after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box} |
|||
html,body{font-size:100%} |
|||
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto;tab-size:4;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased} |
|||
a:hover{cursor:pointer} |
|||
img,object,embed{max-width:100%;height:auto} |
|||
object,embed{height:100%} |
|||
img{-ms-interpolation-mode:bicubic} |
|||
.left{float:left!important} |
|||
.right{float:right!important} |
|||
.text-left{text-align:left!important} |
|||
.text-right{text-align:right!important} |
|||
.text-center{text-align:center!important} |
|||
.text-justify{text-align:justify!important} |
|||
.hide{display:none} |
|||
img,object,svg{display:inline-block;vertical-align:middle} |
|||
textarea{height:auto;min-height:50px} |
|||
select{width:100%} |
|||
.center{margin-left:auto;margin-right:auto} |
|||
.stretch{width:100%} |
|||
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em} |
|||
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr} |
|||
a{color:#2156a5;text-decoration:underline;line-height:inherit} |
|||
a:hover,a:focus{color:#1d4b8f} |
|||
a img{border:0} |
|||
p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility} |
|||
p aside{font-size:.875em;line-height:1.35;font-style:italic} |
|||
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em} |
|||
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0} |
|||
h1{font-size:2.125em} |
|||
h2{font-size:1.6875em} |
|||
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em} |
|||
h4,h5{font-size:1.125em} |
|||
h6{font-size:1em} |
|||
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0} |
|||
em,i{font-style:italic;line-height:inherit} |
|||
strong,b{font-weight:bold;line-height:inherit} |
|||
small{font-size:60%;line-height:inherit} |
|||
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)} |
|||
ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit} |
|||
ul,ol{margin-left:1.5em} |
|||
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em} |
|||
ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit} |
|||
ul.square{list-style-type:square} |
|||
ul.circle{list-style-type:circle} |
|||
ul.disc{list-style-type:disc} |
|||
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0} |
|||
dl dt{margin-bottom:.3125em;font-weight:bold} |
|||
dl dd{margin-bottom:1.25em} |
|||
abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help} |
|||
abbr{text-transform:none} |
|||
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd} |
|||
blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)} |
|||
blockquote cite::before{content:"\2014 \0020"} |
|||
blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)} |
|||
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)} |
|||
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2} |
|||
h1{font-size:2.75em} |
|||
h2{font-size:2.3125em} |
|||
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em} |
|||
h4{font-size:1.4375em}} |
|||
table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede} |
|||
table thead,table tfoot{background:#f7f8f7} |
|||
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left} |
|||
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)} |
|||
table tr.even,table tr.alt{background:#f8f8f7} |
|||
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6} |
|||
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em} |
|||
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400} |
|||
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table} |
|||
.clearfix::after,.float-group::after{clear:both} |
|||
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed;word-wrap:break-word} |
|||
:not(pre)>code.nobreak{word-wrap:normal} |
|||
:not(pre)>code.nowrap{white-space:nowrap} |
|||
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed} |
|||
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit} |
|||
pre>code{display:block} |
|||
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal} |
|||
em em{font-style:normal} |
|||
strong strong{font-weight:400} |
|||
.keyseq{color:rgba(51,51,51,.8)} |
|||
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap} |
|||
.keyseq kbd:first-child{margin-left:0} |
|||
.keyseq kbd:last-child{margin-right:0} |
|||
.menuseq,.menuref{color:#000} |
|||
.menuseq b:not(.caret),.menuref{font-weight:inherit} |
|||
.menuseq{word-spacing:-.02em} |
|||
.menuseq b.caret{font-size:1.25em;line-height:.8} |
|||
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em} |
|||
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400} |
|||
b.button::before{content:"[";padding:0 3px 0 2px} |
|||
b.button::after{content:"]";padding:0 2px 0 3px} |
|||
p a>code:hover{color:rgba(0,0,0,.9)} |
|||
#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em} |
|||
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table} |
|||
#header::after,#content::after,#footnotes::after,#footer::after{clear:both} |
|||
#content{margin-top:1.25em} |
|||
#content::before{content:none} |
|||
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0} |
|||
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf} |
|||
#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px} |
|||
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap} |
|||
#header .details span:first-child{margin-left:-.125em} |
|||
#header .details span.email a{color:rgba(0,0,0,.85)} |
|||
#header .details br{display:none} |
|||
#header .details br+span::before{content:"\00a0\2013\00a0"} |
|||
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)} |
|||
#header .details br+span#revremark::before{content:"\00a0|\00a0"} |
|||
#header #revnumber{text-transform:capitalize} |
|||
#header #revnumber::after{content:"\00a0"} |
|||
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem} |
|||
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em} |
|||
#toc>ul{margin-left:.125em} |
|||
#toc ul.sectlevel0>li>a{font-style:italic} |
|||
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0} |
|||
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none} |
|||
#toc li{line-height:1.3334;margin-top:.3334em} |
|||
#toc a{text-decoration:none} |
|||
#toc a:active{text-decoration:underline} |
|||
#toctitle{color:#7a2518;font-size:1.2em} |
|||
@media screen and (min-width:768px){#toctitle{font-size:1.375em} |
|||
body.toc2{padding-left:15em;padding-right:0} |
|||
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto} |
|||
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em} |
|||
#toc.toc2>ul{font-size:.9em;margin-bottom:0} |
|||
#toc.toc2 ul ul{margin-left:0;padding-left:1em} |
|||
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em} |
|||
body.toc2.toc-right{padding-left:0;padding-right:15em} |
|||
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}} |
|||
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0} |
|||
#toc.toc2{width:20em} |
|||
#toc.toc2 #toctitle{font-size:1.375em} |
|||
#toc.toc2>ul{font-size:.95em} |
|||
#toc.toc2 ul ul{padding-left:1.25em} |
|||
body.toc2.toc-right{padding-left:0;padding-right:20em}} |
|||
#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} |
|||
#content #toc>:first-child{margin-top:0} |
|||
#content #toc>:last-child{margin-bottom:0} |
|||
#footer{max-width:100%;background:rgba(0,0,0,.8);padding:1.25em} |
|||
#footer-text{color:rgba(255,255,255,.8);line-height:1.44} |
|||
#content{margin-bottom:.625em} |
|||
.sect1{padding-bottom:.625em} |
|||
@media screen and (min-width:768px){#content{margin-bottom:1.25em} |
|||
.sect1{padding-bottom:1.25em}} |
|||
.sect1:last-child{padding-bottom:0} |
|||
.sect1+.sect1{border-top:1px solid #e7e7e9} |
|||
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400} |
|||
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em} |
|||
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible} |
|||
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none} |
|||
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221} |
|||
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em} |
|||
details>summary:first-of-type{cursor:pointer;display:list-item;outline:none;margin-bottom:.75em} |
|||
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic} |
|||
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0} |
|||
.paragraph.lead>p,#preamble>.sectionbody>[class="paragraph"]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)} |
|||
table.tableblock #preamble>.sectionbody>[class="paragraph"]:first-of-type p{font-size:inherit} |
|||
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%} |
|||
.admonitionblock>table td.icon{text-align:center;width:80px} |
|||
.admonitionblock>table td.icon img{max-width:none} |
|||
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase} |
|||
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6)} |
|||
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0} |
|||
.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px} |
|||
.exampleblock>.content>:first-child{margin-top:0} |
|||
.exampleblock>.content>:last-child{margin-bottom:0} |
|||
.sidebarblock{border-style:solid;border-width:1px;border-color:#dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;-webkit-border-radius:4px;border-radius:4px} |
|||
.sidebarblock>:first-child{margin-top:0} |
|||
.sidebarblock>:last-child{margin-bottom:0} |
|||
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center} |
|||
.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0} |
|||
.literalblock pre,.listingblock>.content>pre{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;overflow-x:auto;padding:1em;font-size:.8125em} |
|||
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}} |
|||
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}} |
|||
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class="highlight"],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8} |
|||
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)} |
|||
.listingblock>.content{position:relative} |
|||
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5} |
|||
.listingblock:hover code[data-lang]::before{display:block} |
|||
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5} |
|||
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"} |
|||
.listingblock pre.highlightjs{padding:0} |
|||
.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px} |
|||
.listingblock pre.prettyprint{border-width:0} |
|||
.prettyprint{background:#f7f7f8} |
|||
pre.prettyprint .linenums{line-height:1.45;margin-left:2em} |
|||
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0} |
|||
pre.prettyprint li code[data-lang]::before{opacity:1} |
|||
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none} |
|||
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none} |
|||
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal} |
|||
table.linenotable td.code{padding-left:.75em} |
|||
table.linenotable td.linenos{border-right:1px solid currentColor;opacity:.35;padding-right:.5em} |
|||
pre.pygments .lineno{border-right:1px solid currentColor;opacity:.35;display:inline-block;margin-right:.75em} |
|||
pre.pygments .lineno::before{content:"";margin-right:-.125em} |
|||
.quoteblock{margin:0 1em 1.25em 1.5em;display:table} |
|||
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em} |
|||
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify} |
|||
.quoteblock blockquote{margin:0;padding:0;border:0} |
|||
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)} |
|||
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0} |
|||
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right} |
|||
.verseblock{margin:0 1em 1.25em} |
|||
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility} |
|||
.verseblock pre strong{font-weight:400} |
|||
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex} |
|||
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic} |
|||
.quoteblock .attribution br,.verseblock .attribution br{display:none} |
|||
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)} |
|||
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none} |
|||
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0} |
|||
.quoteblock.abstract{margin:0 1em 1.25em;display:block} |
|||
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center} |
|||
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf} |
|||
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0} |
|||
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem} |
|||
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;text-align:left;margin-right:0} |
|||
table.tableblock{max-width:100%;border-collapse:separate} |
|||
p.tableblock:last-child{margin-bottom:0} |
|||
td.tableblock>.content>:last-child{margin-bottom:-1.25em} |
|||
td.tableblock>.content>:last-child.sidebarblock{margin-bottom:0} |
|||
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede} |
|||
table.grid-all>thead>tr>.tableblock,table.grid-all>tbody>tr>.tableblock{border-width:0 1px 1px 0} |
|||
table.grid-all>tfoot>tr>.tableblock{border-width:1px 1px 0 0} |
|||
table.grid-cols>*>tr>.tableblock{border-width:0 1px 0 0} |
|||
table.grid-rows>thead>tr>.tableblock,table.grid-rows>tbody>tr>.tableblock{border-width:0 0 1px} |
|||
table.grid-rows>tfoot>tr>.tableblock{border-width:1px 0 0} |
|||
table.grid-all>*>tr>.tableblock:last-child,table.grid-cols>*>tr>.tableblock:last-child{border-right-width:0} |
|||
table.grid-all>tbody>tr:last-child>.tableblock,table.grid-all>thead:last-child>tr>.tableblock,table.grid-rows>tbody>tr:last-child>.tableblock,table.grid-rows>thead:last-child>tr>.tableblock{border-bottom-width:0} |
|||
table.frame-all{border-width:1px} |
|||
table.frame-sides{border-width:0 1px} |
|||
table.frame-topbot,table.frame-ends{border-width:1px 0} |
|||
table.stripes-all tr,table.stripes-odd tr:nth-of-type(odd),table.stripes-even tr:nth-of-type(even),table.stripes-hover tr:hover{background:#f8f8f7} |
|||
th.halign-left,td.halign-left{text-align:left} |
|||
th.halign-right,td.halign-right{text-align:right} |
|||
th.halign-center,td.halign-center{text-align:center} |
|||
th.valign-top,td.valign-top{vertical-align:top} |
|||
th.valign-bottom,td.valign-bottom{vertical-align:bottom} |
|||
th.valign-middle,td.valign-middle{vertical-align:middle} |
|||
table thead th,table tfoot th{font-weight:bold} |
|||
tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7} |
|||
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold} |
|||
p.tableblock>code:only-child{background:none;padding:0} |
|||
p.tableblock{font-size:1em} |
|||
ol{margin-left:1.75em} |
|||
ul li ol{margin-left:1.5em} |
|||
dl dd{margin-left:1.125em} |
|||
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0} |
|||
ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em} |
|||
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none} |
|||
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em} |
|||
ul.unstyled,ol.unstyled{margin-left:0} |
|||
ul.checklist{margin-left:.625em} |
|||
ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em} |
|||
ul.checklist li>p:first-child>input[type="checkbox"]:first-child{margin-right:.25em} |
|||
ul.inline{display:-ms-flexbox;display:-webkit-box;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em} |
|||
ul.inline>li{margin-left:1.25em} |
|||
.unstyled dl dt{font-weight:400;font-style:normal} |
|||
ol.arabic{list-style-type:decimal} |
|||
ol.decimal{list-style-type:decimal-leading-zero} |
|||
ol.loweralpha{list-style-type:lower-alpha} |
|||
ol.upperalpha{list-style-type:upper-alpha} |
|||
ol.lowerroman{list-style-type:lower-roman} |
|||
ol.upperroman{list-style-type:upper-roman} |
|||
ol.lowergreek{list-style-type:lower-greek} |
|||
.hdlist>table,.colist>table{border:0;background:none} |
|||
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none} |
|||
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em} |
|||
td.hdlist1{font-weight:bold;padding-bottom:1.25em} |
|||
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em} |
|||
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top} |
|||
.colist td:not([class]):first-child img{max-width:none} |
|||
.colist td:not([class]):last-child{padding:.25em 0} |
|||
.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd} |
|||
.imageblock.left{margin:.25em .625em 1.25em 0} |
|||
.imageblock.right{margin:.25em 0 1.25em .625em} |
|||
.imageblock>.title{margin-bottom:0} |
|||
.imageblock.thumb,.imageblock.th{border-width:6px} |
|||
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em} |
|||
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0} |
|||
.image.left{margin-right:.625em} |
|||
.image.right{margin-left:.625em} |
|||
a.image{text-decoration:none;display:inline-block} |
|||
a.image object{pointer-events:none} |
|||
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super} |
|||
sup.footnote a,sup.footnoteref a{text-decoration:none} |
|||
sup.footnote a:active,sup.footnoteref a:active{text-decoration:underline} |
|||
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em} |
|||
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0} |
|||
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em} |
|||
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em} |
|||
#footnotes .footnote:last-of-type{margin-bottom:0} |
|||
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0} |
|||
.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0} |
|||
.gist .file-data>table td.line-data{width:99%} |
|||
div.unbreakable{page-break-inside:avoid} |
|||
.big{font-size:larger} |
|||
.small{font-size:smaller} |
|||
.underline{text-decoration:underline} |
|||
.overline{text-decoration:overline} |
|||
.line-through{text-decoration:line-through} |
|||
.aqua{color:#00bfbf} |
|||
.aqua-background{background:#00fafa} |
|||
.black{color:#000} |
|||
.black-background{background:#000} |
|||
.blue{color:#0000bf} |
|||
.blue-background{background:#0000fa} |
|||
.fuchsia{color:#bf00bf} |
|||
.fuchsia-background{background:#fa00fa} |
|||
.gray{color:#606060} |
|||
.gray-background{background:#7d7d7d} |
|||
.green{color:#006000} |
|||
.green-background{background:#007d00} |
|||
.lime{color:#00bf00} |
|||
.lime-background{background:#00fa00} |
|||
.maroon{color:#600000} |
|||
.maroon-background{background:#7d0000} |
|||
.navy{color:#000060} |
|||
.navy-background{background:#00007d} |
|||
.olive{color:#606000} |
|||
.olive-background{background:#7d7d00} |
|||
.purple{color:#600060} |
|||
.purple-background{background:#7d007d} |
|||
.red{color:#bf0000} |
|||
.red-background{background:#fa0000} |
|||
.silver{color:#909090} |
|||
.silver-background{background:#bcbcbc} |
|||
.teal{color:#006060} |
|||
.teal-background{background:#007d7d} |
|||
.white{color:#bfbfbf} |
|||
.white-background{background:#fafafa} |
|||
.yellow{color:#bfbf00} |
|||
.yellow-background{background:#fafa00} |
|||
span.icon>.fa{cursor:default} |
|||
a span.icon>.fa{cursor:inherit} |
|||
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default} |
|||
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c} |
|||
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111} |
|||
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900} |
|||
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400} |
|||
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000} |
|||
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold} |
|||
.conum[data-value] *{color:#fff!important} |
|||
.conum[data-value]+b{display:none} |
|||
.conum[data-value]::after{content:attr(data-value)} |
|||
pre .conum[data-value]{position:relative;top:-.125em} |
|||
b.conum *{color:inherit!important} |
|||
.conum:not([data-value]):empty{display:none} |
|||
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility} |
|||
h1,h2,p,td.content,span.alt{letter-spacing:-.01em} |
|||
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em} |
|||
p,blockquote,dt,td.content,span.alt{font-size:1.0625rem} |
|||
p{margin-bottom:1.25rem} |
|||
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em} |
|||
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc} |
|||
.print-only{display:none!important} |
|||
@page{margin:1.25cm .75cm} |
|||
@media print{*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important} |
|||
html{font-size:80%} |
|||
a{color:inherit!important;text-decoration:underline!important} |
|||
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important} |
|||
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em} |
|||
abbr[title]::after{content:" (" attr(title) ")"} |
|||
pre,blockquote,tr,img,object,svg{page-break-inside:avoid} |
|||
thead{display:table-header-group} |
|||
svg{max-width:100%} |
|||
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3} |
|||
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid} |
|||
#toc,.sidebarblock,.exampleblock>.content{background:none!important} |
|||
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important} |
|||
body.book #header{text-align:center} |
|||
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em} |
|||
body.book #header .details{border:0!important;display:block;padding:0!important} |
|||
body.book #header .details span:first-child{margin-left:0!important} |
|||
body.book #header .details br{display:block} |
|||
body.book #header .details br+span::before{content:none!important} |
|||
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important} |
|||
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always} |
|||
.listingblock code[data-lang]::before{display:block} |
|||
#footer{padding:0 .9375em} |
|||
.hide-on-print{display:none!important} |
|||
.print-only{display:block!important} |
|||
.hide-for-print{display:none!important} |
|||
.show-for-print{display:inherit!important}} |
|||
@media print,amzn-kf8{#header>h1:first-child{margin-top:1.25rem} |
|||
.sect1{padding:0!important} |
|||
.sect1+.sect1{border:0} |
|||
#footer{background:none} |
|||
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}} |
|||
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}} |
|||
</style> |
|||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css"> |
|||
</head> |
|||
<body class="article toc2 toc-right"> |
|||
<div id="header"> |
|||
<h1>Swarm deploy Yaml !</h1> |
|||
<div class="details"> |
|||
<span id="author" class="author">Apostolos rootApostolos@swarmlab.io</span><br> |
|||
</div> |
|||
<div id="toc" class="toc2"> |
|||
<div id="toctitle">Πίνακας περιεχομένων</div> |
|||
<ul class="sectlevel1"> |
|||
<li><a href="#_create_swarm_service_yaml">1. Create Swarm Service YAML</a></li> |
|||
</ul> |
|||
</div> |
|||
</div> |
|||
<div id="content"> |
|||
<div id="preamble"> |
|||
<div class="sectionbody"> |
|||
<div class="paragraph"> |
|||
<p><br></p> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
<div class="sect1"> |
|||
<h2 id="_create_swarm_service_yaml">1. Create Swarm Service YAML</h2> |
|||
<div class="sectionbody"> |
|||
<div class="listingblock"> |
|||
<div class="title">YAML</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"><span class="key">version</span>: <span class="string"><span class="delimiter">"</span><span class="content">3.4</span><span class="delimiter">"</span></span> |
|||
|
|||
<span class="key">services</span>: |
|||
|
|||
<span class="comment"># όνομα υπηρεσίας</span> |
|||
<span class="key">master</span>: |
|||
<span class="key">image</span>: <span class="string"><span class="content">registry.vlabs.uniwa.gr:5080/swarmlab-service-mpi2</span></span> |
|||
<span class="key">user</span>: <span class="string"><span class="content">root</span></span> |
|||
<span class="comment"># ENTRYPOINT instruction allows you to configure a container that will run as an executable.</span> |
|||
<span class="key">entrypoint</span>: <span class="string"><span class="content">["mpi_bootstrap", "role=master", "mpi_master_service_name=master", "mpi_worker_service_name=worker"]</span></span> |
|||
<span class="comment"># Environment variables (declared with the ENV statement) can also be used in certain instructions as variables to be interpreted by the Dockerfile.</span> |
|||
<span class="comment"># https://docs.docker.com/engine/reference/builder/#environment-replacement</span> |
|||
<span class="key">environment</span>: |
|||
- <span class="string"><span class="content">PASSWORD=padatest</span></span> |
|||
- <span class="string"><span class="content">PASSWORDVIEW=padatestview</span></span> |
|||
- <span class="string"><span class="content">SERVERROLE=master</span></span> |
|||
- <span class="string"><span class="content">SERVERWEB=no</span></span> |
|||
<span class="comment"># docker service inspect ondemand_mpi2_master</span> |
|||
- <span class="string"><span class="content">NODENAME={{.Node.Hostname}}</span></span> |
|||
- <span class="string"><span class="content">NODEID={{.Node.ID}}</span></span> |
|||
- <span class="string"><span class="content">SERVICEID={{.Service.ID}}</span></span> |
|||
- <span class="string"><span class="content">SERVICENAME={{.Service.Name}}</span></span> |
|||
- <span class="string"><span class="content">TASKID={{.Task.ID}}</span></span> |
|||
- <span class="string"><span class="content">TASKNAME={{.Task.Name}}</span></span> |
|||
- <span class="string"><span class="content">TASKREPID={{.Task.Slot}}</span></span> |
|||
<span class="comment"># Specify configuration related to the deployment and running of services.</span> |
|||
<span class="key">deploy</span>: |
|||
<span class="comment"># If the service is replicated (which is the default), specify the number of containers that should be running at any given time.</span> |
|||
<span class="comment"># In global mode, running one replica of service per swarm node. The number of global replicas is equal to the number of swarm nodes. In replica mode, you can run any number of service instances.</span> |
|||
<span class="key">replicas</span>: <span class="string"><span class="content">9</span></span> |
|||
<span class="key">placement</span>: |
|||
<span class="comment"># max_replicas_per_node: 1</span> |
|||
<span class="key">constraints</span>: |
|||
- <span class="string"><span class="content">node.role == worker</span></span> |
|||
<span class="comment"># Configures resource constraints.</span> |
|||
<span class="comment"># resources:</span> |
|||
<span class="comment"># limits:</span> |
|||
<span class="comment"># cpus: '0.50'</span> |
|||
<span class="comment"># memory: 500M</span> |
|||
<span class="comment"># reservations:</span> |
|||
<span class="comment"># cpus: '0.25'</span> |
|||
|
|||
|
|||
|
|||
|
|||
<span class="comment"># memory: 200M</span> |
|||
<span class="comment"># Configures how the service should be rollbacked in case of a failing update.</span> |
|||
<span class="comment">#</span> |
|||
<span class="comment"># parallelism: The number of containers to rollback at a time. If set to 0, all containers rollback simultaneously.</span> |
|||
<span class="comment"># delay: The time to wait between each container group’s rollback (default 0s).</span> |
|||
<span class="comment"># failure_action: What to do if a rollback fails. One of continue or pause (default pause)</span> |
|||
<span class="comment"># monitor: Duration after each task update to monitor for failure (ns|us|ms|s|m|h) (default 0s).</span> |
|||
<span class="comment"># max_failure_ratio: Failure rate to tolerate during a rollback (default 0).</span> |
|||
<span class="comment"># order: Order of operations during rollbacks. One of stop-first (old task is stopped before starting new one), or start-first (new task is started first, and the running tasks briefly overlap) (default stop-first).</span> |
|||
<span class="comment">#</span> |
|||
<span class="key">restart_policy</span>: |
|||
<span class="key">condition</span>: <span class="string"><span class="content">on-failure</span></span> |
|||
<span class="key">delay</span>: <span class="string"><span class="content">5s</span></span> |
|||
<span class="key">max_attempts</span>: <span class="string"><span class="content">5</span></span> |
|||
<span class="key">window</span>: <span class="string"><span class="content">120s</span></span> |
|||
|
|||
<span class="comment">#Configures how the service should be updated. Useful for configuring rolling updates.</span> |
|||
<span class="comment">#</span> |
|||
<span class="comment"># parallelism: The number of containers to update at a time.</span> |
|||
<span class="comment"># delay: The time to wait between updating a group of containers.</span> |
|||
<span class="comment"># failure_action: What to do if an update fails. One of continue, rollback, or pause (default: pause).</span> |
|||
<span class="comment"># monitor: Duration after each task update to monitor for failure (ns|us|ms|s|m|h) (default 0s).</span> |
|||
<span class="comment"># max_failure_ratio: Failure rate to tolerate during an update.</span> |
|||
<span class="comment"># order: Order of operations during updates. One of stop-first (old task is stopped before starting new one), or start-first (new task is started first, and the running tasks briefly overlap) (default stop-first) Note: Only supported for v3.4 and higher.</span> |
|||
<span class="comment">#</span> |
|||
<span class="comment">#</span> |
|||
<span class="key">update_config</span>: |
|||
<span class="key">parallelism</span>: <span class="string"><span class="content">2</span></span> |
|||
<span class="key">delay</span>: <span class="string"><span class="content">10s</span></span> |
|||
<span class="key">order</span>: <span class="string"><span class="content">stop-first</span></span> |
|||
<span class="key">networks</span>: |
|||
<span class="key">mpi2-net</span>: |
|||
<span class="key">volumes</span>: |
|||
- <span class="string"><span class="content">mpi3_vol:/var/share</span></span> |
|||
<span class="key">ports</span>: |
|||
- <span class="string"><span class="delimiter">"</span><span class="content">55520:80</span><span class="delimiter">"</span></span> |
|||
- <span class="string"><span class="delimiter">"</span><span class="content">55521:8088</span><span class="delimiter">"</span></span> |
|||
- <span class="string"><span class="delimiter">"</span><span class="content">55522:6088</span><span class="delimiter">"</span></span> |
|||
- <span class="string"><span class="delimiter">"</span><span class="content">55523:6080</span><span class="delimiter">"</span></span> |
|||
|
|||
|
|||
<span class="key">worker</span>: |
|||
<span class="key">image</span>: <span class="string"><span class="content">registry.vlabs.uniwa.gr:5080/swarmlab-service-mpi2</span></span> |
|||
<span class="key">user</span>: <span class="string"><span class="content">root</span></span> |
|||
<span class="key">entrypoint</span>: <span class="string"><span class="content">["mpi_bootstrap", "role=worker", "mpi_master_service_name=master", "mpi_worker_service_name=worker"]</span></span> |
|||
<span class="key">environment</span>: |
|||
- <span class="string"><span class="content">SERVERROLE=worker</span></span> |
|||
- <span class="string"><span class="content">SERVERWEB=no</span></span> |
|||
- <span class="string"><span class="content">NODENAME={{.Node.Hostname}}</span></span> |
|||
- <span class="string"><span class="content">NODEID={{.Node.ID}}</span></span> |
|||
- <span class="string"><span class="content">SERVICEID={{.Service.ID}}</span></span> |
|||
- <span class="string"><span class="content">SERVICENAME={{.Service.Name}}</span></span> |
|||
- <span class="string"><span class="content">TASKID={{.Task.ID}}</span></span> |
|||
- <span class="string"><span class="content">TASKNAME={{.Task.Name}}</span></span> |
|||
- <span class="string"><span class="content">TASKREPID={{.Task.Slot}}</span></span> |
|||
<span class="key">deploy</span>: |
|||
<span class="key">replicas</span>: <span class="string"><span class="content">5</span></span> |
|||
<span class="key">placement</span>: |
|||
<span class="comment"># max_replicas_per_node: 1</span> |
|||
<span class="key">constraints</span>: |
|||
- <span class="string"><span class="content">node.role == worker</span></span> |
|||
<span class="comment">#- node.id == ${worker}</span> |
|||
<span class="comment"># resources:</span> |
|||
<span class="comment"># limits:</span> |
|||
<span class="comment"># cpus: '0.50'</span> |
|||
<span class="comment"># memory: 500M</span> |
|||
<span class="comment"># reservations:</span> |
|||
<span class="comment"># cpus: '0.25'</span> |
|||
<span class="comment"># memory: 200M</span> |
|||
<span class="key">restart_policy</span>: |
|||
<span class="key">condition</span>: <span class="string"><span class="content">on-failure</span></span> |
|||
<span class="key">delay</span>: <span class="string"><span class="content">5s</span></span> |
|||
<span class="key">max_attempts</span>: <span class="string"><span class="content">5</span></span> |
|||
<span class="key">window</span>: <span class="string"><span class="content">120s</span></span> |
|||
<span class="key">update_config</span>: |
|||
<span class="key">parallelism</span>: <span class="string"><span class="content">2</span></span> |
|||
<span class="key">delay</span>: <span class="string"><span class="content">10s</span></span> |
|||
<span class="key">order</span>: <span class="string"><span class="content">stop-first</span></span> |
|||
<span class="key">networks</span>: |
|||
<span class="key">mpi2-net</span>: |
|||
<span class="key">volumes</span>: |
|||
- <span class="string"><span class="content">mpi3_vol:/var/share</span></span> |
|||
|
|||
<span class="key">web</span>: |
|||
<span class="key">image</span>: <span class="string"><span class="content">registry.vlabs.uniwa.gr:5080/swarmlab-service-sshfs</span></span> |
|||
<span class="key">user</span>: <span class="string"><span class="content">root</span></span> |
|||
<span class="key">entrypoint</span>: <span class="string"><span class="content">["mpi_bootstrap", "role=worker", "mpi_master_service_name=master", "mpi_worker_service_name=worker"]</span></span> |
|||
<span class="key">environment</span>: |
|||
- <span class="string"><span class="content">SERVERROLE=worker</span></span> |
|||
- <span class="string"><span class="content">SERVERWEB=yes</span></span> |
|||
- <span class="string"><span class="content">NODENAME={{.Node.Hostname}}</span></span> |
|||
- <span class="string"><span class="content">NODEID={{.Node.ID}}</span></span> |
|||
- <span class="string"><span class="content">SERVICEID={{.Service.ID}}</span></span> |
|||
- <span class="string"><span class="content">SERVICENAME={{.Service.Name}}</span></span> |
|||
- <span class="string"><span class="content">TASKID={{.Task.ID}}</span></span> |
|||
- <span class="string"><span class="content">TASKNAME={{.Task.Name}}</span></span> |
|||
- <span class="string"><span class="content">TASKREPID={{.Task.Slot}}</span></span> |
|||
<span class="key">deploy</span>: |
|||
<span class="key">replicas</span>: <span class="string"><span class="content">1</span></span> |
|||
<span class="key">placement</span>: |
|||
<span class="key">constraints</span>: |
|||
- <span class="string"><span class="content">node.role == worker</span></span> |
|||
<span class="key">resources</span>: |
|||
<span class="key">limits</span>: |
|||
<span class="key">cpus</span>: <span class="string"><span class="content">'0.50'</span></span> |
|||
<span class="key">memory</span>: <span class="string"><span class="content">500M</span></span> |
|||
<span class="key">reservations</span>: |
|||
<span class="key">cpus</span>: <span class="string"><span class="content">'0.25'</span></span> |
|||
<span class="key">memory</span>: <span class="string"><span class="content">200M</span></span> |
|||
<span class="key">restart_policy</span>: |
|||
<span class="key">condition</span>: <span class="string"><span class="content">on-failure</span></span> |
|||
<span class="key">delay</span>: <span class="string"><span class="content">5s</span></span> |
|||
<span class="key">max_attempts</span>: <span class="string"><span class="content">5</span></span> |
|||
<span class="key">window</span>: <span class="string"><span class="content">120s</span></span> |
|||
<span class="key">update_config</span>: |
|||
<span class="key">parallelism</span>: <span class="string"><span class="content">2</span></span> |
|||
<span class="key">delay</span>: <span class="string"><span class="content">10s</span></span> |
|||
<span class="key">order</span>: <span class="string"><span class="content">stop-first</span></span> |
|||
<span class="key">networks</span>: |
|||
<span class="key">mpi2-net</span>: |
|||
<span class="key">ports</span>: |
|||
- <span class="string"><span class="delimiter">"</span><span class="content">55519:80</span><span class="delimiter">"</span></span> |
|||
|
|||
<span class="comment">#Creates a new network. The DRIVER accepts bridge or overlay which are the built-in network drivers.</span> |
|||
<span class="comment">#Bridge networks are isolated networks on a single Engine installation. If you want to create a network that spans multiple Docker hosts each running an Engine, you must create an overlay network. Unlike bridge networks, overlay networks require some pre-existing conditions before you can create one.</span> |
|||
<span class="comment">#</span> |
|||
<span class="key">networks</span>: |
|||
<span class="key">mpi2-net</span>: |
|||
|
|||
|
|||
<span class="comment">#Mount host paths or named volumes</span> |
|||
<span class="comment">#Creates a new volume that containers can consume and store data in.</span> |
|||
<span class="key">volumes</span>: |
|||
<span class="key">mpi3_vol</span>: |
|||
<span class="key">external</span>: <span class="string"><span class="content">false</span></span></code></pre> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
<div id="footer"> |
|||
<div id="footer-text"> |
|||
Last updated 2021-04-05 18:27:02 UTC |
|||
</div> |
|||
</div> |
|||
<style> |
|||
/* Stylesheet for CodeRay to match GitHub theme | MIT License | http://foundation.zurb.com */ |
|||
pre.CodeRay{background:#f7f7f8} |
|||
.CodeRay .line-numbers{border-right:1px solid currentColor;opacity:.35;padding:0 .5em 0 0} |
|||
.CodeRay span.line-numbers{display:inline-block;margin-right:.75em} |
|||
.CodeRay .line-numbers strong{color:#000} |
|||
table.CodeRay{border-collapse:separate;border:0;margin-bottom:0;background:none} |
|||
table.CodeRay td{vertical-align:top;line-height:inherit} |
|||
table.CodeRay td.line-numbers{text-align:right} |
|||
table.CodeRay td.code{padding:0 0 0 .75em} |
|||
.CodeRay .debug{color:#fff !important;background:#000080 !important} |
|||
.CodeRay .annotation{color:#007} |
|||
.CodeRay .attribute-name{color:#000080} |
|||
.CodeRay .attribute-value{color:#700} |
|||
.CodeRay .binary{color:#509} |
|||
.CodeRay .comment{color:#998;font-style:italic} |
|||
.CodeRay .char{color:#04d} |
|||
.CodeRay .char .content{color:#04d} |
|||
.CodeRay .char .delimiter{color:#039} |
|||
.CodeRay .class{color:#458;font-weight:bold} |
|||
.CodeRay .complex{color:#a08} |
|||
.CodeRay .constant,.CodeRay .predefined-constant{color:#008080} |
|||
.CodeRay .color{color:#099} |
|||
.CodeRay .class-variable{color:#369} |
|||
.CodeRay .decorator{color:#b0b} |
|||
.CodeRay .definition{color:#099} |
|||
.CodeRay .delimiter{color:#000} |
|||
.CodeRay .doc{color:#970} |
|||
.CodeRay .doctype{color:#34b} |
|||
.CodeRay .doc-string{color:#d42} |
|||
.CodeRay .escape{color:#666} |
|||
.CodeRay .entity{color:#800} |
|||
.CodeRay .error{color:#808} |
|||
.CodeRay .exception{color:inherit} |
|||
.CodeRay .filename{color:#099} |
|||
.CodeRay .function{color:#900;font-weight:bold} |
|||
.CodeRay .global-variable{color:#008080} |
|||
.CodeRay .hex{color:#058} |
|||
.CodeRay .integer,.CodeRay .float{color:#099} |
|||
.CodeRay .include{color:#555} |
|||
.CodeRay .inline{color:#000} |
|||
.CodeRay .inline .inline{background:#ccc} |
|||
.CodeRay .inline .inline .inline{background:#bbb} |
|||
.CodeRay .inline .inline-delimiter{color:#d14} |
|||
.CodeRay .inline-delimiter{color:#d14} |
|||
.CodeRay .important{color:#555;font-weight:bold} |
|||
.CodeRay .interpreted{color:#b2b} |
|||
.CodeRay .instance-variable{color:#008080} |
|||
.CodeRay .label{color:#970} |
|||
.CodeRay .local-variable{color:#963} |
|||
.CodeRay .octal{color:#40e} |
|||
.CodeRay .predefined{color:#369} |
|||
.CodeRay .preprocessor{color:#579} |
|||
.CodeRay .pseudo-class{color:#555} |
|||
.CodeRay .directive{font-weight:bold} |
|||
.CodeRay .type{font-weight:bold} |
|||
.CodeRay .predefined-type{color:inherit} |
|||
.CodeRay .reserved,.CodeRay .keyword {color:#000;font-weight:bold} |
|||
.CodeRay .key{color:#808} |
|||
.CodeRay .key .delimiter{color:#606} |
|||
.CodeRay .key .char{color:#80f} |
|||
.CodeRay .value{color:#088} |
|||
.CodeRay .regexp .delimiter{color:#808} |
|||
.CodeRay .regexp .content{color:#808} |
|||
.CodeRay .regexp .modifier{color:#808} |
|||
.CodeRay .regexp .char{color:#d14} |
|||
.CodeRay .regexp .function{color:#404;font-weight:bold} |
|||
.CodeRay .string{color:#d20} |
|||
.CodeRay .string .string .string{background:#ffd0d0} |
|||
.CodeRay .string .content{color:#d14} |
|||
.CodeRay .string .char{color:#d14} |
|||
.CodeRay .string .delimiter{color:#d14} |
|||
.CodeRay .shell{color:#d14} |
|||
.CodeRay .shell .delimiter{color:#d14} |
|||
.CodeRay .symbol{color:#990073} |
|||
.CodeRay .symbol .content{color:#a60} |
|||
.CodeRay .symbol .delimiter{color:#630} |
|||
.CodeRay .tag{color:#008080} |
|||
.CodeRay .tag-special{color:#d70} |
|||
.CodeRay .variable{color:#036} |
|||
.CodeRay .insert{background:#afa} |
|||
.CodeRay .delete{background:#faa} |
|||
.CodeRay .change{color:#aaf;background:#007} |
|||
.CodeRay .head{color:#f8f;background:#505} |
|||
.CodeRay .insert .insert{color:#080} |
|||
.CodeRay .delete .delete{color:#800} |
|||
.CodeRay .change .change{color:#66f} |
|||
.CodeRay .head .head{color:#f4f} |
|||
</style> |
|||
</body> |
|||
</html> |
|||
File diff suppressed because it is too large
|
After Width: | Height: | Size: 15 KiB |
|
After Width: | Height: | Size: 48 KiB |
|
After Width: | Height: | Size: 85 KiB |
|
After Width: | Height: | Size: 125 KiB |
@ -0,0 +1,166 @@ |
|||
= Swarm Minio HowTo! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
:description: IoT Εισαγωγή στο Cloud |
|||
:keywords: Cloud, swarm |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Πίνακας περιεχομένων |
|||
:toclevels: 4 |
|||
:source-highlighter: coderay |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
{empty} + |
|||
|
|||
== configure Minio |
|||
|
|||
=== Connect to Server |
|||
|
|||
.create policy |
|||
[source,yaml] |
|||
---- |
|||
mc config host add --insecure [SESSION] https://ip:9443 key secret --api s3v4 |
|||
e.g. |
|||
mc config host add --insecure mysession https://83.212.119.119:9443 mykey mysecret --api s3v4 |
|||
---- |
|||
|
|||
=== Create policy |
|||
|
|||
.create Default policy file: test.json |
|||
[source,yaml] |
|||
---- |
|||
{ |
|||
"Version": "2012-10-17", |
|||
"Statement": [ |
|||
{ |
|||
"Effect": "Allow", |
|||
"Principal": { |
|||
"AWS": [ |
|||
"" |
|||
] |
|||
}, |
|||
"Action": [ |
|||
"s3:GetBucketLocation", // <1> |
|||
"s3:HeadBucket", // <1> |
|||
"s3:ListBucket", // <1> |
|||
"s3:ListBucketMultipartUploads" // <1> |
|||
], |
|||
"Resource": [ |
|||
"arn:aws:s3:::test" // <2> |
|||
] |
|||
}, |
|||
{ |
|||
"Effect": "Allow", |
|||
"Principal": { |
|||
"AWS": "*" |
|||
}, |
|||
"Action": [ |
|||
"s3:DeleteObject", // <3> |
|||
"s3:GetObject", // <3> |
|||
"s3:ListMultipartUploadParts", // <3> |
|||
"s3:PutObject" // <3> |
|||
], |
|||
"Resource": [ |
|||
"arn:aws:s3:::test/*" // <4> |
|||
] |
|||
} |
|||
] |
|||
} |
|||
---- |
|||
<1> Bucket properties https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html[more info^] |
|||
<2> Bucket source |
|||
<3> Object properties https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-objects[more info^] |
|||
<3> Object source |
|||
|
|||
|
|||
=== Apply policy |
|||
|
|||
.create policy |
|||
[source,yaml] |
|||
---- |
|||
#!/bin/bash |
|||
|
|||
mc mb --insecure mymsession/test // <1> |
|||
|
|||
mc admin policy add --insecure mymisession test ./test.json // <2> |
|||
|
|||
mc admin user add --insecure mymsession usertest bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd // <3> |
|||
|
|||
mc admin policy set --insecure mymsession test user=usertest // <4> |
|||
---- |
|||
<1> create bucket |
|||
<2> Apply policy |
|||
<3> create user |
|||
<4> Apply policy User2bucket |
|||
|
|||
|
|||
=== Sync local2Server |
|||
|
|||
.sync local2server |
|||
[source,yaml] |
|||
---- |
|||
export MINIO_MIRROR_PARAMETERS="--delete" |
|||
export MINIO_SERVER_URL="ip:9443" // <1> |
|||
export MINIO_PROT=https |
|||
export MINIO_SERVER_DOCUMENTROOT="/home/user/syncdir" // <2> |
|||
export MINIO_WEB_BUCKET=test1 |
|||
export MINIO_WEB_PASS="bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd" |
|||
export AWS_CONFIG_FILE="$HOME/.aws/config" |
|||
export AWS_ACCESS_KEY_ID=$MINIO_WEB_BUCKET |
|||
export AWS_SECRET_ACCESS_KEY=$MINIO_WEB_PASS |
|||
|
|||
aws --endpoint-url $MINIO_PROT://$MINIO_SERVER_URL s3 sync MINIO_SERVER_DOCUMENTROOT s3://$MINIO_WEB_BUCKET $MINIO_MIRROR_PARAMETERS |
|||
---- |
|||
<1> Minio Server ip |
|||
<2> Directory sync |
|||
|
|||
|
|||
|
|||
=== Sync Server2Local |
|||
|
|||
.sync server2local |
|||
[source,yaml] |
|||
---- |
|||
export MINIO_MIRROR_PARAMETERS="--delete" |
|||
export MINIO_SERVER_URL="ip:9443" // <1> |
|||
export MINIO_PROT=https |
|||
export MINIO_SERVER_DOCUMENTROOT="/var/www/html" // <2> |
|||
export MINIO_WEB_BUCKET=test |
|||
export MINIO_WEB_PASS="bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd" |
|||
export AWS_CONFIG_FILE="$HOME/.aws/config" |
|||
export AWS_ACCESS_KEY_ID=$MINIO_WEB_BUCKET |
|||
export AWS_SECRET_ACCESS_KEY=$MINIO_WEB_PASS |
|||
|
|||
aws --endpoint-url $MINIO_PROT://$MINIO_SERVER_URL s3 sync s3://$MINIO_WEB_BUCKET $MINIO_SERVER_DOCUMENTROOT $MINIO_MIRROR_PARAMETERS |
|||
---- |
|||
<1> Minio Server ip |
|||
<2> Directory sync |
|||
|
|||
|
|||
=== Docker-compose env |
|||
|
|||
.sync docker-compose env |
|||
[source,yaml] |
|||
---- |
|||
environment: //<1> |
|||
- MINIO_MIRROR_PARAMETERS= --delete |
|||
- MINIO_SERVER_URL=ip:9443 |
|||
- MINIO_PROT=https |
|||
- MINIO_SERVER_DOCUMENTROOT=/var/www/html |
|||
- MINIO_WEB_BUCKET=test |
|||
- MINIO_WEB_PASS=bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd |
|||
- AWS_CONFIG_FILE=/home/user/.aws/config |
|||
---- |
|||
<1> add env to docker-comose |
|||
|
|||
|
|||
=== Automating sync with cron |
|||
|
|||
.crond file |
|||
[source,yaml] |
|||
---- |
|||
*/15 * * * * root /PATH/local2server >> /var/log/usercron.log 2>&1 / <1> |
|||
---- |
|||
<1> Every 15 minutes |
|||
@ -0,0 +1,791 @@ |
|||
<!DOCTYPE html> |
|||
<html lang="en"> |
|||
<head> |
|||
<meta charset="UTF-8"> |
|||
<meta http-equiv="X-UA-Compatible" content="IE=edge"> |
|||
<meta name="viewport" content="width=device-width, initial-scale=1.0"> |
|||
<meta name="generator" content="Asciidoctor 2.0.10"> |
|||
<meta name="description" content="IoT Εισαγωγή στο Cloud"> |
|||
<meta name="keywords" content="Cloud, swarm"> |
|||
<meta name="author" content="Apostolos rootApostolos@swarmlab.io"> |
|||
<title>Swarm Minio HowTo!</title> |
|||
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700"> |
|||
<style> |
|||
/* Asciidoctor default stylesheet | MIT License | https://asciidoctor.org */ |
|||
/* Uncomment @import statement to use as custom stylesheet */ |
|||
/*@import "https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700";*/ |
|||
article,aside,details,figcaption,figure,footer,header,hgroup,main,nav,section{display:block} |
|||
audio,video{display:inline-block} |
|||
audio:not([controls]){display:none;height:0} |
|||
html{font-family:sans-serif;-ms-text-size-adjust:100%;-webkit-text-size-adjust:100%} |
|||
a{background:none} |
|||
a:focus{outline:thin dotted} |
|||
a:active,a:hover{outline:0} |
|||
h1{font-size:2em;margin:.67em 0} |
|||
abbr[title]{border-bottom:1px dotted} |
|||
b,strong{font-weight:bold} |
|||
dfn{font-style:italic} |
|||
hr{-moz-box-sizing:content-box;box-sizing:content-box;height:0} |
|||
mark{background:#ff0;color:#000} |
|||
code,kbd,pre,samp{font-family:monospace;font-size:1em} |
|||
pre{white-space:pre-wrap} |
|||
q{quotes:"\201C" "\201D" "\2018" "\2019"} |
|||
small{font-size:80%} |
|||
sub,sup{font-size:75%;line-height:0;position:relative;vertical-align:baseline} |
|||
sup{top:-.5em} |
|||
sub{bottom:-.25em} |
|||
img{border:0} |
|||
svg:not(:root){overflow:hidden} |
|||
figure{margin:0} |
|||
fieldset{border:1px solid silver;margin:0 2px;padding:.35em .625em .75em} |
|||
legend{border:0;padding:0} |
|||
button,input,select,textarea{font-family:inherit;font-size:100%;margin:0} |
|||
button,input{line-height:normal} |
|||
button,select{text-transform:none} |
|||
button,html input[type="button"],input[type="reset"],input[type="submit"]{-webkit-appearance:button;cursor:pointer} |
|||
button[disabled],html input[disabled]{cursor:default} |
|||
input[type="checkbox"],input[type="radio"]{box-sizing:border-box;padding:0} |
|||
button::-moz-focus-inner,input::-moz-focus-inner{border:0;padding:0} |
|||
textarea{overflow:auto;vertical-align:top} |
|||
table{border-collapse:collapse;border-spacing:0} |
|||
*,*::before,*::after{-moz-box-sizing:border-box;-webkit-box-sizing:border-box;box-sizing:border-box} |
|||
html,body{font-size:100%} |
|||
body{background:#fff;color:rgba(0,0,0,.8);padding:0;margin:0;font-family:"Noto Serif","DejaVu Serif",serif;font-weight:400;font-style:normal;line-height:1;position:relative;cursor:auto;tab-size:4;-moz-osx-font-smoothing:grayscale;-webkit-font-smoothing:antialiased} |
|||
a:hover{cursor:pointer} |
|||
img,object,embed{max-width:100%;height:auto} |
|||
object,embed{height:100%} |
|||
img{-ms-interpolation-mode:bicubic} |
|||
.left{float:left!important} |
|||
.right{float:right!important} |
|||
.text-left{text-align:left!important} |
|||
.text-right{text-align:right!important} |
|||
.text-center{text-align:center!important} |
|||
.text-justify{text-align:justify!important} |
|||
.hide{display:none} |
|||
img,object,svg{display:inline-block;vertical-align:middle} |
|||
textarea{height:auto;min-height:50px} |
|||
select{width:100%} |
|||
.center{margin-left:auto;margin-right:auto} |
|||
.stretch{width:100%} |
|||
.subheader,.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{line-height:1.45;color:#7a2518;font-weight:400;margin-top:0;margin-bottom:.25em} |
|||
div,dl,dt,dd,ul,ol,li,h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6,pre,form,p,blockquote,th,td{margin:0;padding:0;direction:ltr} |
|||
a{color:#2156a5;text-decoration:underline;line-height:inherit} |
|||
a:hover,a:focus{color:#1d4b8f} |
|||
a img{border:0} |
|||
p{font-family:inherit;font-weight:400;font-size:1em;line-height:1.6;margin-bottom:1.25em;text-rendering:optimizeLegibility} |
|||
p aside{font-size:.875em;line-height:1.35;font-style:italic} |
|||
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{font-family:"Open Sans","DejaVu Sans",sans-serif;font-weight:300;font-style:normal;color:#ba3925;text-rendering:optimizeLegibility;margin-top:1em;margin-bottom:.5em;line-height:1.0125em} |
|||
h1 small,h2 small,h3 small,#toctitle small,.sidebarblock>.content>.title small,h4 small,h5 small,h6 small{font-size:60%;color:#e99b8f;line-height:0} |
|||
h1{font-size:2.125em} |
|||
h2{font-size:1.6875em} |
|||
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.375em} |
|||
h4,h5{font-size:1.125em} |
|||
h6{font-size:1em} |
|||
hr{border:solid #dddddf;border-width:1px 0 0;clear:both;margin:1.25em 0 1.1875em;height:0} |
|||
em,i{font-style:italic;line-height:inherit} |
|||
strong,b{font-weight:bold;line-height:inherit} |
|||
small{font-size:60%;line-height:inherit} |
|||
code{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;font-weight:400;color:rgba(0,0,0,.9)} |
|||
ul,ol,dl{font-size:1em;line-height:1.6;margin-bottom:1.25em;list-style-position:outside;font-family:inherit} |
|||
ul,ol{margin-left:1.5em} |
|||
ul li ul,ul li ol{margin-left:1.25em;margin-bottom:0;font-size:1em} |
|||
ul.square li ul,ul.circle li ul,ul.disc li ul{list-style:inherit} |
|||
ul.square{list-style-type:square} |
|||
ul.circle{list-style-type:circle} |
|||
ul.disc{list-style-type:disc} |
|||
ol li ul,ol li ol{margin-left:1.25em;margin-bottom:0} |
|||
dl dt{margin-bottom:.3125em;font-weight:bold} |
|||
dl dd{margin-bottom:1.25em} |
|||
abbr,acronym{text-transform:uppercase;font-size:90%;color:rgba(0,0,0,.8);border-bottom:1px dotted #ddd;cursor:help} |
|||
abbr{text-transform:none} |
|||
blockquote{margin:0 0 1.25em;padding:.5625em 1.25em 0 1.1875em;border-left:1px solid #ddd} |
|||
blockquote cite{display:block;font-size:.9375em;color:rgba(0,0,0,.6)} |
|||
blockquote cite::before{content:"\2014 \0020"} |
|||
blockquote cite a,blockquote cite a:visited{color:rgba(0,0,0,.6)} |
|||
blockquote,blockquote p{line-height:1.6;color:rgba(0,0,0,.85)} |
|||
@media screen and (min-width:768px){h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2} |
|||
h1{font-size:2.75em} |
|||
h2{font-size:2.3125em} |
|||
h3,#toctitle,.sidebarblock>.content>.title{font-size:1.6875em} |
|||
h4{font-size:1.4375em}} |
|||
table{background:#fff;margin-bottom:1.25em;border:solid 1px #dedede} |
|||
table thead,table tfoot{background:#f7f8f7} |
|||
table thead tr th,table thead tr td,table tfoot tr th,table tfoot tr td{padding:.5em .625em .625em;font-size:inherit;color:rgba(0,0,0,.8);text-align:left} |
|||
table tr th,table tr td{padding:.5625em .625em;font-size:inherit;color:rgba(0,0,0,.8)} |
|||
table tr.even,table tr.alt{background:#f8f8f7} |
|||
table thead tr th,table tfoot tr th,table tbody tr td,table tr td,table tfoot tr td{display:table-cell;line-height:1.6} |
|||
h1,h2,h3,#toctitle,.sidebarblock>.content>.title,h4,h5,h6{line-height:1.2;word-spacing:-.05em} |
|||
h1 strong,h2 strong,h3 strong,#toctitle strong,.sidebarblock>.content>.title strong,h4 strong,h5 strong,h6 strong{font-weight:400} |
|||
.clearfix::before,.clearfix::after,.float-group::before,.float-group::after{content:" ";display:table} |
|||
.clearfix::after,.float-group::after{clear:both} |
|||
:not(pre):not([class^=L])>code{font-size:.9375em;font-style:normal!important;letter-spacing:0;padding:.1em .5ex;word-spacing:-.15em;background:#f7f7f8;-webkit-border-radius:4px;border-radius:4px;line-height:1.45;text-rendering:optimizeSpeed;word-wrap:break-word} |
|||
:not(pre)>code.nobreak{word-wrap:normal} |
|||
:not(pre)>code.nowrap{white-space:nowrap} |
|||
pre{color:rgba(0,0,0,.9);font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;line-height:1.45;text-rendering:optimizeSpeed} |
|||
pre code,pre pre{color:inherit;font-size:inherit;line-height:inherit} |
|||
pre>code{display:block} |
|||
pre.nowrap,pre.nowrap pre{white-space:pre;word-wrap:normal} |
|||
em em{font-style:normal} |
|||
strong strong{font-weight:400} |
|||
.keyseq{color:rgba(51,51,51,.8)} |
|||
kbd{font-family:"Droid Sans Mono","DejaVu Sans Mono",monospace;display:inline-block;color:rgba(0,0,0,.8);font-size:.65em;line-height:1.45;background:#f7f7f7;border:1px solid #ccc;-webkit-border-radius:3px;border-radius:3px;-webkit-box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em white inset;box-shadow:0 1px 0 rgba(0,0,0,.2),0 0 0 .1em #fff inset;margin:0 .15em;padding:.2em .5em;vertical-align:middle;position:relative;top:-.1em;white-space:nowrap} |
|||
.keyseq kbd:first-child{margin-left:0} |
|||
.keyseq kbd:last-child{margin-right:0} |
|||
.menuseq,.menuref{color:#000} |
|||
.menuseq b:not(.caret),.menuref{font-weight:inherit} |
|||
.menuseq{word-spacing:-.02em} |
|||
.menuseq b.caret{font-size:1.25em;line-height:.8} |
|||
.menuseq i.caret{font-weight:bold;text-align:center;width:.45em} |
|||
b.button::before,b.button::after{position:relative;top:-1px;font-weight:400} |
|||
b.button::before{content:"[";padding:0 3px 0 2px} |
|||
b.button::after{content:"]";padding:0 2px 0 3px} |
|||
p a>code:hover{color:rgba(0,0,0,.9)} |
|||
#header,#content,#footnotes,#footer{width:100%;margin-left:auto;margin-right:auto;margin-top:0;margin-bottom:0;max-width:62.5em;*zoom:1;position:relative;padding-left:.9375em;padding-right:.9375em} |
|||
#header::before,#header::after,#content::before,#content::after,#footnotes::before,#footnotes::after,#footer::before,#footer::after{content:" ";display:table} |
|||
#header::after,#content::after,#footnotes::after,#footer::after{clear:both} |
|||
#content{margin-top:1.25em} |
|||
#content::before{content:none} |
|||
#header>h1:first-child{color:rgba(0,0,0,.85);margin-top:2.25rem;margin-bottom:0} |
|||
#header>h1:first-child+#toc{margin-top:8px;border-top:1px solid #dddddf} |
|||
#header>h1:only-child,body.toc2 #header>h1:nth-last-child(2){border-bottom:1px solid #dddddf;padding-bottom:8px} |
|||
#header .details{border-bottom:1px solid #dddddf;line-height:1.45;padding-top:.25em;padding-bottom:.25em;padding-left:.25em;color:rgba(0,0,0,.6);display:-ms-flexbox;display:-webkit-flex;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap} |
|||
#header .details span:first-child{margin-left:-.125em} |
|||
#header .details span.email a{color:rgba(0,0,0,.85)} |
|||
#header .details br{display:none} |
|||
#header .details br+span::before{content:"\00a0\2013\00a0"} |
|||
#header .details br+span.author::before{content:"\00a0\22c5\00a0";color:rgba(0,0,0,.85)} |
|||
#header .details br+span#revremark::before{content:"\00a0|\00a0"} |
|||
#header #revnumber{text-transform:capitalize} |
|||
#header #revnumber::after{content:"\00a0"} |
|||
#content>h1:first-child:not([class]){color:rgba(0,0,0,.85);border-bottom:1px solid #dddddf;padding-bottom:8px;margin-top:0;padding-top:1rem;margin-bottom:1.25rem} |
|||
#toc{border-bottom:1px solid #e7e7e9;padding-bottom:.5em} |
|||
#toc>ul{margin-left:.125em} |
|||
#toc ul.sectlevel0>li>a{font-style:italic} |
|||
#toc ul.sectlevel0 ul.sectlevel1{margin:.5em 0} |
|||
#toc ul{font-family:"Open Sans","DejaVu Sans",sans-serif;list-style-type:none} |
|||
#toc li{line-height:1.3334;margin-top:.3334em} |
|||
#toc a{text-decoration:none} |
|||
#toc a:active{text-decoration:underline} |
|||
#toctitle{color:#7a2518;font-size:1.2em} |
|||
@media screen and (min-width:768px){#toctitle{font-size:1.375em} |
|||
body.toc2{padding-left:15em;padding-right:0} |
|||
#toc.toc2{margin-top:0!important;background:#f8f8f7;position:fixed;width:15em;left:0;top:0;border-right:1px solid #e7e7e9;border-top-width:0!important;border-bottom-width:0!important;z-index:1000;padding:1.25em 1em;height:100%;overflow:auto} |
|||
#toc.toc2 #toctitle{margin-top:0;margin-bottom:.8rem;font-size:1.2em} |
|||
#toc.toc2>ul{font-size:.9em;margin-bottom:0} |
|||
#toc.toc2 ul ul{margin-left:0;padding-left:1em} |
|||
#toc.toc2 ul.sectlevel0 ul.sectlevel1{padding-left:0;margin-top:.5em;margin-bottom:.5em} |
|||
body.toc2.toc-right{padding-left:0;padding-right:15em} |
|||
body.toc2.toc-right #toc.toc2{border-right-width:0;border-left:1px solid #e7e7e9;left:auto;right:0}} |
|||
@media screen and (min-width:1280px){body.toc2{padding-left:20em;padding-right:0} |
|||
#toc.toc2{width:20em} |
|||
#toc.toc2 #toctitle{font-size:1.375em} |
|||
#toc.toc2>ul{font-size:.95em} |
|||
#toc.toc2 ul ul{padding-left:1.25em} |
|||
body.toc2.toc-right{padding-left:0;padding-right:20em}} |
|||
#content #toc{border-style:solid;border-width:1px;border-color:#e0e0dc;margin-bottom:1.25em;padding:1.25em;background:#f8f8f7;-webkit-border-radius:4px;border-radius:4px} |
|||
#content #toc>:first-child{margin-top:0} |
|||
#content #toc>:last-child{margin-bottom:0} |
|||
#footer{max-width:100%;background:rgba(0,0,0,.8);padding:1.25em} |
|||
#footer-text{color:rgba(255,255,255,.8);line-height:1.44} |
|||
#content{margin-bottom:.625em} |
|||
.sect1{padding-bottom:.625em} |
|||
@media screen and (min-width:768px){#content{margin-bottom:1.25em} |
|||
.sect1{padding-bottom:1.25em}} |
|||
.sect1:last-child{padding-bottom:0} |
|||
.sect1+.sect1{border-top:1px solid #e7e7e9} |
|||
#content h1>a.anchor,h2>a.anchor,h3>a.anchor,#toctitle>a.anchor,.sidebarblock>.content>.title>a.anchor,h4>a.anchor,h5>a.anchor,h6>a.anchor{position:absolute;z-index:1001;width:1.5ex;margin-left:-1.5ex;display:block;text-decoration:none!important;visibility:hidden;text-align:center;font-weight:400} |
|||
#content h1>a.anchor::before,h2>a.anchor::before,h3>a.anchor::before,#toctitle>a.anchor::before,.sidebarblock>.content>.title>a.anchor::before,h4>a.anchor::before,h5>a.anchor::before,h6>a.anchor::before{content:"\00A7";font-size:.85em;display:block;padding-top:.1em} |
|||
#content h1:hover>a.anchor,#content h1>a.anchor:hover,h2:hover>a.anchor,h2>a.anchor:hover,h3:hover>a.anchor,#toctitle:hover>a.anchor,.sidebarblock>.content>.title:hover>a.anchor,h3>a.anchor:hover,#toctitle>a.anchor:hover,.sidebarblock>.content>.title>a.anchor:hover,h4:hover>a.anchor,h4>a.anchor:hover,h5:hover>a.anchor,h5>a.anchor:hover,h6:hover>a.anchor,h6>a.anchor:hover{visibility:visible} |
|||
#content h1>a.link,h2>a.link,h3>a.link,#toctitle>a.link,.sidebarblock>.content>.title>a.link,h4>a.link,h5>a.link,h6>a.link{color:#ba3925;text-decoration:none} |
|||
#content h1>a.link:hover,h2>a.link:hover,h3>a.link:hover,#toctitle>a.link:hover,.sidebarblock>.content>.title>a.link:hover,h4>a.link:hover,h5>a.link:hover,h6>a.link:hover{color:#a53221} |
|||
details,.audioblock,.imageblock,.literalblock,.listingblock,.stemblock,.videoblock{margin-bottom:1.25em} |
|||
details>summary:first-of-type{cursor:pointer;display:list-item;outline:none;margin-bottom:.75em} |
|||
.admonitionblock td.content>.title,.audioblock>.title,.exampleblock>.title,.imageblock>.title,.listingblock>.title,.literalblock>.title,.stemblock>.title,.openblock>.title,.paragraph>.title,.quoteblock>.title,table.tableblock>.title,.verseblock>.title,.videoblock>.title,.dlist>.title,.olist>.title,.ulist>.title,.qlist>.title,.hdlist>.title{text-rendering:optimizeLegibility;text-align:left;font-family:"Noto Serif","DejaVu Serif",serif;font-size:1rem;font-style:italic} |
|||
table.tableblock.fit-content>caption.title{white-space:nowrap;width:0} |
|||
.paragraph.lead>p,#preamble>.sectionbody>[class="paragraph"]:first-of-type p{font-size:1.21875em;line-height:1.6;color:rgba(0,0,0,.85)} |
|||
table.tableblock #preamble>.sectionbody>[class="paragraph"]:first-of-type p{font-size:inherit} |
|||
.admonitionblock>table{border-collapse:separate;border:0;background:none;width:100%} |
|||
.admonitionblock>table td.icon{text-align:center;width:80px} |
|||
.admonitionblock>table td.icon img{max-width:none} |
|||
.admonitionblock>table td.icon .title{font-weight:bold;font-family:"Open Sans","DejaVu Sans",sans-serif;text-transform:uppercase} |
|||
.admonitionblock>table td.content{padding-left:1.125em;padding-right:1.25em;border-left:1px solid #dddddf;color:rgba(0,0,0,.6)} |
|||
.admonitionblock>table td.content>:last-child>:last-child{margin-bottom:0} |
|||
.exampleblock>.content{border-style:solid;border-width:1px;border-color:#e6e6e6;margin-bottom:1.25em;padding:1.25em;background:#fff;-webkit-border-radius:4px;border-radius:4px} |
|||
.exampleblock>.content>:first-child{margin-top:0} |
|||
.exampleblock>.content>:last-child{margin-bottom:0} |
|||
.sidebarblock{border-style:solid;border-width:1px;border-color:#dbdbd6;margin-bottom:1.25em;padding:1.25em;background:#f3f3f2;-webkit-border-radius:4px;border-radius:4px} |
|||
.sidebarblock>:first-child{margin-top:0} |
|||
.sidebarblock>:last-child{margin-bottom:0} |
|||
.sidebarblock>.content>.title{color:#7a2518;margin-top:0;text-align:center} |
|||
.exampleblock>.content>:last-child>:last-child,.exampleblock>.content .olist>ol>li:last-child>:last-child,.exampleblock>.content .ulist>ul>li:last-child>:last-child,.exampleblock>.content .qlist>ol>li:last-child>:last-child,.sidebarblock>.content>:last-child>:last-child,.sidebarblock>.content .olist>ol>li:last-child>:last-child,.sidebarblock>.content .ulist>ul>li:last-child>:last-child,.sidebarblock>.content .qlist>ol>li:last-child>:last-child{margin-bottom:0} |
|||
.literalblock pre,.listingblock>.content>pre{-webkit-border-radius:4px;border-radius:4px;word-wrap:break-word;overflow-x:auto;padding:1em;font-size:.8125em} |
|||
@media screen and (min-width:768px){.literalblock pre,.listingblock>.content>pre{font-size:.90625em}} |
|||
@media screen and (min-width:1280px){.literalblock pre,.listingblock>.content>pre{font-size:1em}} |
|||
.literalblock pre,.listingblock>.content>pre:not(.highlight),.listingblock>.content>pre[class="highlight"],.listingblock>.content>pre[class^="highlight "]{background:#f7f7f8} |
|||
.literalblock.output pre{color:#f7f7f8;background:rgba(0,0,0,.9)} |
|||
.listingblock>.content{position:relative} |
|||
.listingblock code[data-lang]::before{display:none;content:attr(data-lang);position:absolute;font-size:.75em;top:.425rem;right:.5rem;line-height:1;text-transform:uppercase;color:inherit;opacity:.5} |
|||
.listingblock:hover code[data-lang]::before{display:block} |
|||
.listingblock.terminal pre .command::before{content:attr(data-prompt);padding-right:.5em;color:inherit;opacity:.5} |
|||
.listingblock.terminal pre .command:not([data-prompt])::before{content:"$"} |
|||
.listingblock pre.highlightjs{padding:0} |
|||
.listingblock pre.highlightjs>code{padding:1em;-webkit-border-radius:4px;border-radius:4px} |
|||
.listingblock pre.prettyprint{border-width:0} |
|||
.prettyprint{background:#f7f7f8} |
|||
pre.prettyprint .linenums{line-height:1.45;margin-left:2em} |
|||
pre.prettyprint li{background:none;list-style-type:inherit;padding-left:0} |
|||
pre.prettyprint li code[data-lang]::before{opacity:1} |
|||
pre.prettyprint li:not(:first-child) code[data-lang]::before{display:none} |
|||
table.linenotable{border-collapse:separate;border:0;margin-bottom:0;background:none} |
|||
table.linenotable td[class]{color:inherit;vertical-align:top;padding:0;line-height:inherit;white-space:normal} |
|||
table.linenotable td.code{padding-left:.75em} |
|||
table.linenotable td.linenos{border-right:1px solid currentColor;opacity:.35;padding-right:.5em} |
|||
pre.pygments .lineno{border-right:1px solid currentColor;opacity:.35;display:inline-block;margin-right:.75em} |
|||
pre.pygments .lineno::before{content:"";margin-right:-.125em} |
|||
.quoteblock{margin:0 1em 1.25em 1.5em;display:table} |
|||
.quoteblock:not(.excerpt)>.title{margin-left:-1.5em;margin-bottom:.75em} |
|||
.quoteblock blockquote,.quoteblock p{color:rgba(0,0,0,.85);font-size:1.15rem;line-height:1.75;word-spacing:.1em;letter-spacing:0;font-style:italic;text-align:justify} |
|||
.quoteblock blockquote{margin:0;padding:0;border:0} |
|||
.quoteblock blockquote::before{content:"\201c";float:left;font-size:2.75em;font-weight:bold;line-height:.6em;margin-left:-.6em;color:#7a2518;text-shadow:0 1px 2px rgba(0,0,0,.1)} |
|||
.quoteblock blockquote>.paragraph:last-child p{margin-bottom:0} |
|||
.quoteblock .attribution{margin-top:.75em;margin-right:.5ex;text-align:right} |
|||
.verseblock{margin:0 1em 1.25em} |
|||
.verseblock pre{font-family:"Open Sans","DejaVu Sans",sans;font-size:1.15rem;color:rgba(0,0,0,.85);font-weight:300;text-rendering:optimizeLegibility} |
|||
.verseblock pre strong{font-weight:400} |
|||
.verseblock .attribution{margin-top:1.25rem;margin-left:.5ex} |
|||
.quoteblock .attribution,.verseblock .attribution{font-size:.9375em;line-height:1.45;font-style:italic} |
|||
.quoteblock .attribution br,.verseblock .attribution br{display:none} |
|||
.quoteblock .attribution cite,.verseblock .attribution cite{display:block;letter-spacing:-.025em;color:rgba(0,0,0,.6)} |
|||
.quoteblock.abstract blockquote::before,.quoteblock.excerpt blockquote::before,.quoteblock .quoteblock blockquote::before{display:none} |
|||
.quoteblock.abstract blockquote,.quoteblock.abstract p,.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{line-height:1.6;word-spacing:0} |
|||
.quoteblock.abstract{margin:0 1em 1.25em;display:block} |
|||
.quoteblock.abstract>.title{margin:0 0 .375em;font-size:1.15em;text-align:center} |
|||
.quoteblock.excerpt>blockquote,.quoteblock .quoteblock{padding:0 0 .25em 1em;border-left:.25em solid #dddddf} |
|||
.quoteblock.excerpt,.quoteblock .quoteblock{margin-left:0} |
|||
.quoteblock.excerpt blockquote,.quoteblock.excerpt p,.quoteblock .quoteblock blockquote,.quoteblock .quoteblock p{color:inherit;font-size:1.0625rem} |
|||
.quoteblock.excerpt .attribution,.quoteblock .quoteblock .attribution{color:inherit;text-align:left;margin-right:0} |
|||
table.tableblock{max-width:100%;border-collapse:separate} |
|||
p.tableblock:last-child{margin-bottom:0} |
|||
td.tableblock>.content>:last-child{margin-bottom:-1.25em} |
|||
td.tableblock>.content>:last-child.sidebarblock{margin-bottom:0} |
|||
table.tableblock,th.tableblock,td.tableblock{border:0 solid #dedede} |
|||
table.grid-all>thead>tr>.tableblock,table.grid-all>tbody>tr>.tableblock{border-width:0 1px 1px 0} |
|||
table.grid-all>tfoot>tr>.tableblock{border-width:1px 1px 0 0} |
|||
table.grid-cols>*>tr>.tableblock{border-width:0 1px 0 0} |
|||
table.grid-rows>thead>tr>.tableblock,table.grid-rows>tbody>tr>.tableblock{border-width:0 0 1px} |
|||
table.grid-rows>tfoot>tr>.tableblock{border-width:1px 0 0} |
|||
table.grid-all>*>tr>.tableblock:last-child,table.grid-cols>*>tr>.tableblock:last-child{border-right-width:0} |
|||
table.grid-all>tbody>tr:last-child>.tableblock,table.grid-all>thead:last-child>tr>.tableblock,table.grid-rows>tbody>tr:last-child>.tableblock,table.grid-rows>thead:last-child>tr>.tableblock{border-bottom-width:0} |
|||
table.frame-all{border-width:1px} |
|||
table.frame-sides{border-width:0 1px} |
|||
table.frame-topbot,table.frame-ends{border-width:1px 0} |
|||
table.stripes-all tr,table.stripes-odd tr:nth-of-type(odd),table.stripes-even tr:nth-of-type(even),table.stripes-hover tr:hover{background:#f8f8f7} |
|||
th.halign-left,td.halign-left{text-align:left} |
|||
th.halign-right,td.halign-right{text-align:right} |
|||
th.halign-center,td.halign-center{text-align:center} |
|||
th.valign-top,td.valign-top{vertical-align:top} |
|||
th.valign-bottom,td.valign-bottom{vertical-align:bottom} |
|||
th.valign-middle,td.valign-middle{vertical-align:middle} |
|||
table thead th,table tfoot th{font-weight:bold} |
|||
tbody tr th{display:table-cell;line-height:1.6;background:#f7f8f7} |
|||
tbody tr th,tbody tr th p,tfoot tr th,tfoot tr th p{color:rgba(0,0,0,.8);font-weight:bold} |
|||
p.tableblock>code:only-child{background:none;padding:0} |
|||
p.tableblock{font-size:1em} |
|||
ol{margin-left:1.75em} |
|||
ul li ol{margin-left:1.5em} |
|||
dl dd{margin-left:1.125em} |
|||
dl dd:last-child,dl dd:last-child>:last-child{margin-bottom:0} |
|||
ol>li p,ul>li p,ul dd,ol dd,.olist .olist,.ulist .ulist,.ulist .olist,.olist .ulist{margin-bottom:.625em} |
|||
ul.checklist,ul.none,ol.none,ul.no-bullet,ol.no-bullet,ol.unnumbered,ul.unstyled,ol.unstyled{list-style-type:none} |
|||
ul.no-bullet,ol.no-bullet,ol.unnumbered{margin-left:.625em} |
|||
ul.unstyled,ol.unstyled{margin-left:0} |
|||
ul.checklist{margin-left:.625em} |
|||
ul.checklist li>p:first-child>.fa-square-o:first-child,ul.checklist li>p:first-child>.fa-check-square-o:first-child{width:1.25em;font-size:.8em;position:relative;bottom:.125em} |
|||
ul.checklist li>p:first-child>input[type="checkbox"]:first-child{margin-right:.25em} |
|||
ul.inline{display:-ms-flexbox;display:-webkit-box;display:flex;-ms-flex-flow:row wrap;-webkit-flex-flow:row wrap;flex-flow:row wrap;list-style:none;margin:0 0 .625em -1.25em} |
|||
ul.inline>li{margin-left:1.25em} |
|||
.unstyled dl dt{font-weight:400;font-style:normal} |
|||
ol.arabic{list-style-type:decimal} |
|||
ol.decimal{list-style-type:decimal-leading-zero} |
|||
ol.loweralpha{list-style-type:lower-alpha} |
|||
ol.upperalpha{list-style-type:upper-alpha} |
|||
ol.lowerroman{list-style-type:lower-roman} |
|||
ol.upperroman{list-style-type:upper-roman} |
|||
ol.lowergreek{list-style-type:lower-greek} |
|||
.hdlist>table,.colist>table{border:0;background:none} |
|||
.hdlist>table>tbody>tr,.colist>table>tbody>tr{background:none} |
|||
td.hdlist1,td.hdlist2{vertical-align:top;padding:0 .625em} |
|||
td.hdlist1{font-weight:bold;padding-bottom:1.25em} |
|||
.literalblock+.colist,.listingblock+.colist{margin-top:-.5em} |
|||
.colist td:not([class]):first-child{padding:.4em .75em 0;line-height:1;vertical-align:top} |
|||
.colist td:not([class]):first-child img{max-width:none} |
|||
.colist td:not([class]):last-child{padding:.25em 0} |
|||
.thumb,.th{line-height:0;display:inline-block;border:solid 4px #fff;-webkit-box-shadow:0 0 0 1px #ddd;box-shadow:0 0 0 1px #ddd} |
|||
.imageblock.left{margin:.25em .625em 1.25em 0} |
|||
.imageblock.right{margin:.25em 0 1.25em .625em} |
|||
.imageblock>.title{margin-bottom:0} |
|||
.imageblock.thumb,.imageblock.th{border-width:6px} |
|||
.imageblock.thumb>.title,.imageblock.th>.title{padding:0 .125em} |
|||
.image.left,.image.right{margin-top:.25em;margin-bottom:.25em;display:inline-block;line-height:0} |
|||
.image.left{margin-right:.625em} |
|||
.image.right{margin-left:.625em} |
|||
a.image{text-decoration:none;display:inline-block} |
|||
a.image object{pointer-events:none} |
|||
sup.footnote,sup.footnoteref{font-size:.875em;position:static;vertical-align:super} |
|||
sup.footnote a,sup.footnoteref a{text-decoration:none} |
|||
sup.footnote a:active,sup.footnoteref a:active{text-decoration:underline} |
|||
#footnotes{padding-top:.75em;padding-bottom:.75em;margin-bottom:.625em} |
|||
#footnotes hr{width:20%;min-width:6.25em;margin:-.25em 0 .75em;border-width:1px 0 0} |
|||
#footnotes .footnote{padding:0 .375em 0 .225em;line-height:1.3334;font-size:.875em;margin-left:1.2em;margin-bottom:.2em} |
|||
#footnotes .footnote a:first-of-type{font-weight:bold;text-decoration:none;margin-left:-1.05em} |
|||
#footnotes .footnote:last-of-type{margin-bottom:0} |
|||
#content #footnotes{margin-top:-.625em;margin-bottom:0;padding:.75em 0} |
|||
.gist .file-data>table{border:0;background:#fff;width:100%;margin-bottom:0} |
|||
.gist .file-data>table td.line-data{width:99%} |
|||
div.unbreakable{page-break-inside:avoid} |
|||
.big{font-size:larger} |
|||
.small{font-size:smaller} |
|||
.underline{text-decoration:underline} |
|||
.overline{text-decoration:overline} |
|||
.line-through{text-decoration:line-through} |
|||
.aqua{color:#00bfbf} |
|||
.aqua-background{background:#00fafa} |
|||
.black{color:#000} |
|||
.black-background{background:#000} |
|||
.blue{color:#0000bf} |
|||
.blue-background{background:#0000fa} |
|||
.fuchsia{color:#bf00bf} |
|||
.fuchsia-background{background:#fa00fa} |
|||
.gray{color:#606060} |
|||
.gray-background{background:#7d7d7d} |
|||
.green{color:#006000} |
|||
.green-background{background:#007d00} |
|||
.lime{color:#00bf00} |
|||
.lime-background{background:#00fa00} |
|||
.maroon{color:#600000} |
|||
.maroon-background{background:#7d0000} |
|||
.navy{color:#000060} |
|||
.navy-background{background:#00007d} |
|||
.olive{color:#606000} |
|||
.olive-background{background:#7d7d00} |
|||
.purple{color:#600060} |
|||
.purple-background{background:#7d007d} |
|||
.red{color:#bf0000} |
|||
.red-background{background:#fa0000} |
|||
.silver{color:#909090} |
|||
.silver-background{background:#bcbcbc} |
|||
.teal{color:#006060} |
|||
.teal-background{background:#007d7d} |
|||
.white{color:#bfbfbf} |
|||
.white-background{background:#fafafa} |
|||
.yellow{color:#bfbf00} |
|||
.yellow-background{background:#fafa00} |
|||
span.icon>.fa{cursor:default} |
|||
a span.icon>.fa{cursor:inherit} |
|||
.admonitionblock td.icon [class^="fa icon-"]{font-size:2.5em;text-shadow:1px 1px 2px rgba(0,0,0,.5);cursor:default} |
|||
.admonitionblock td.icon .icon-note::before{content:"\f05a";color:#19407c} |
|||
.admonitionblock td.icon .icon-tip::before{content:"\f0eb";text-shadow:1px 1px 2px rgba(155,155,0,.8);color:#111} |
|||
.admonitionblock td.icon .icon-warning::before{content:"\f071";color:#bf6900} |
|||
.admonitionblock td.icon .icon-caution::before{content:"\f06d";color:#bf3400} |
|||
.admonitionblock td.icon .icon-important::before{content:"\f06a";color:#bf0000} |
|||
.conum[data-value]{display:inline-block;color:#fff!important;background:rgba(0,0,0,.8);-webkit-border-radius:100px;border-radius:100px;text-align:center;font-size:.75em;width:1.67em;height:1.67em;line-height:1.67em;font-family:"Open Sans","DejaVu Sans",sans-serif;font-style:normal;font-weight:bold} |
|||
.conum[data-value] *{color:#fff!important} |
|||
.conum[data-value]+b{display:none} |
|||
.conum[data-value]::after{content:attr(data-value)} |
|||
pre .conum[data-value]{position:relative;top:-.125em} |
|||
b.conum *{color:inherit!important} |
|||
.conum:not([data-value]):empty{display:none} |
|||
dt,th.tableblock,td.content,div.footnote{text-rendering:optimizeLegibility} |
|||
h1,h2,p,td.content,span.alt{letter-spacing:-.01em} |
|||
p strong,td.content strong,div.footnote strong{letter-spacing:-.005em} |
|||
p,blockquote,dt,td.content,span.alt{font-size:1.0625rem} |
|||
p{margin-bottom:1.25rem} |
|||
.sidebarblock p,.sidebarblock dt,.sidebarblock td.content,p.tableblock{font-size:1em} |
|||
.exampleblock>.content{background:#fffef7;border-color:#e0e0dc;-webkit-box-shadow:0 1px 4px #e0e0dc;box-shadow:0 1px 4px #e0e0dc} |
|||
.print-only{display:none!important} |
|||
@page{margin:1.25cm .75cm} |
|||
@media print{*{-webkit-box-shadow:none!important;box-shadow:none!important;text-shadow:none!important} |
|||
html{font-size:80%} |
|||
a{color:inherit!important;text-decoration:underline!important} |
|||
a.bare,a[href^="#"],a[href^="mailto:"]{text-decoration:none!important} |
|||
a[href^="http:"]:not(.bare)::after,a[href^="https:"]:not(.bare)::after{content:"(" attr(href) ")";display:inline-block;font-size:.875em;padding-left:.25em} |
|||
abbr[title]::after{content:" (" attr(title) ")"} |
|||
pre,blockquote,tr,img,object,svg{page-break-inside:avoid} |
|||
thead{display:table-header-group} |
|||
svg{max-width:100%} |
|||
p,blockquote,dt,td.content{font-size:1em;orphans:3;widows:3} |
|||
h2,h3,#toctitle,.sidebarblock>.content>.title{page-break-after:avoid} |
|||
#toc,.sidebarblock,.exampleblock>.content{background:none!important} |
|||
#toc{border-bottom:1px solid #dddddf!important;padding-bottom:0!important} |
|||
body.book #header{text-align:center} |
|||
body.book #header>h1:first-child{border:0!important;margin:2.5em 0 1em} |
|||
body.book #header .details{border:0!important;display:block;padding:0!important} |
|||
body.book #header .details span:first-child{margin-left:0!important} |
|||
body.book #header .details br{display:block} |
|||
body.book #header .details br+span::before{content:none!important} |
|||
body.book #toc{border:0!important;text-align:left!important;padding:0!important;margin:0!important} |
|||
body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-break-before:always} |
|||
.listingblock code[data-lang]::before{display:block} |
|||
#footer{padding:0 .9375em} |
|||
.hide-on-print{display:none!important} |
|||
.print-only{display:block!important} |
|||
.hide-for-print{display:none!important} |
|||
.show-for-print{display:inherit!important}} |
|||
@media print,amzn-kf8{#header>h1:first-child{margin-top:1.25rem} |
|||
.sect1{padding:0!important} |
|||
.sect1+.sect1{border:0} |
|||
#footer{background:none} |
|||
#footer-text{color:rgba(0,0,0,.6);font-size:.9em}} |
|||
@media amzn-kf8{#header,#content,#footnotes,#footer{padding:0}} |
|||
</style> |
|||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css"> |
|||
</head> |
|||
<body class="article toc2 toc-right"> |
|||
<div id="header"> |
|||
<h1>Swarm Minio HowTo!</h1> |
|||
<div class="details"> |
|||
<span id="author" class="author">Apostolos rootApostolos@swarmlab.io</span><br> |
|||
</div> |
|||
<div id="toc" class="toc2"> |
|||
<div id="toctitle">Πίνακας περιεχομένων</div> |
|||
<ul class="sectlevel1"> |
|||
<li><a href="#_configure_minio">1. configure Minio</a> |
|||
<ul class="sectlevel2"> |
|||
<li><a href="#_connect_to_server">1.1. Connect to Server</a></li> |
|||
<li><a href="#_create_policy">1.2. Create policy</a></li> |
|||
<li><a href="#_apply_policy">1.3. Apply policy</a></li> |
|||
<li><a href="#_sync_local2server">1.4. Sync local2Server</a></li> |
|||
<li><a href="#_sync_server2local">1.5. Sync Server2Local</a></li> |
|||
<li><a href="#_docker_compose_env">1.6. Docker-compose env</a></li> |
|||
<li><a href="#_automating_sync_with_cron">1.7. Automating sync with cron</a></li> |
|||
</ul> |
|||
</li> |
|||
</ul> |
|||
</div> |
|||
</div> |
|||
<div id="content"> |
|||
<div id="preamble"> |
|||
<div class="sectionbody"> |
|||
<div class="paragraph"> |
|||
<p><br></p> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
<div class="sect1"> |
|||
<h2 id="_configure_minio">1. configure Minio</h2> |
|||
<div class="sectionbody"> |
|||
<div class="sect2"> |
|||
<h3 id="_connect_to_server">1.1. Connect to Server</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">create policy</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"><span class="error">mc config host add --insecure [SESSION] https://ip:9443 key secret --api s3v4</span> |
|||
<span class="error">e.g.</span> |
|||
<span class="error">mc config host add --insecure mysession https://83.212.119.119:9443 mykey mysecret --api s3v4</span></code></pre> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
<div class="sect2"> |
|||
<h3 id="_create_policy">1.2. Create policy</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">create Default policy file: test.json</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml">{ |
|||
<span class="key"><span class="delimiter">"</span><span class="content">Version</span><span class="delimiter">"</span></span>: <span class="string"><span class="delimiter">"</span><span class="content">2012-10-17</span><span class="delimiter">"</span></span><span class="string"><span class="content">,</span></span> |
|||
<span class="key"><span class="delimiter">"</span><span class="content">Statement</span><span class="delimiter">"</span></span>: <span class="string"><span class="content">[</span><span class="content"> |
|||
{ |
|||
"Effect": "Allow", |
|||
"Principal": { |
|||
"AWS": [ |
|||
"" |
|||
] |
|||
}, |
|||
"Action": [ |
|||
"s3:GetBucketLocation", <i class="conum" data-value="1"></i><b>(1)</b> |
|||
"s3:HeadBucket", <i class="conum" data-value="1"></i><b>(1)</b> |
|||
"s3:ListBucket", <i class="conum" data-value="1"></i><b>(1)</b> |
|||
"s3:ListBucketMultipartUploads" <i class="conum" data-value="1"></i><b>(1)</b> |
|||
], |
|||
"Resource": [ |
|||
"arn:aws:s3:::test" <i class="conum" data-value="2"></i><b>(2)</b> |
|||
] |
|||
}, |
|||
{ |
|||
"Effect": "Allow", |
|||
"Principal": { |
|||
"AWS": "*" |
|||
}, |
|||
"Action": [ |
|||
"s3:DeleteObject", <i class="conum" data-value="3"></i><b>(3)</b> |
|||
"s3:GetObject", <i class="conum" data-value="3"></i><b>(3)</b> |
|||
"s3:ListMultipartUploadParts", <i class="conum" data-value="3"></i><b>(3)</b> |
|||
"s3:PutObject" <i class="conum" data-value="3"></i><b>(3)</b> |
|||
], |
|||
"Resource": [ |
|||
"arn:aws:s3:::test/*" <i class="conum" data-value="4"></i><b>(4)</b> |
|||
] |
|||
}</span></span> |
|||
] |
|||
}</code></pre> |
|||
</div> |
|||
</div> |
|||
<div class="colist arabic"> |
|||
<table> |
|||
<tr> |
|||
<td><i class="conum" data-value="1"></i><b>1</b></td> |
|||
<td>Bucket properties <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket.html" target="_blank" rel="noopener">more info</a></td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="2"></i><b>2</b></td> |
|||
<td>Bucket source</td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="3"></i><b>3</b></td> |
|||
<td>Object properties <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-objects" target="_blank" rel="noopener">more info</a></td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="4"></i><b>4</b></td> |
|||
<td>Object source</td> |
|||
</tr> |
|||
</table> |
|||
</div> |
|||
</div> |
|||
<div class="sect2"> |
|||
<h3 id="_apply_policy">1.3. Apply policy</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">create policy</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"><span class="comment">#!/bin/bash</span> |
|||
|
|||
<span class="error">mc mb --insecure mymsession/test </span><i class="conum" data-value="1"></i><b>(1)</b> |
|||
|
|||
<span class="error">mc admin policy add --insecure mymisession test ./test.json </span><i class="conum" data-value="2"></i><b>(2)</b> |
|||
|
|||
<span class="error">mc admin user add --insecure mymsession usertest bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd </span><i class="conum" data-value="3"></i><b>(3)</b> |
|||
|
|||
<span class="error">mc admin policy set --insecure mymsession test user=usertest </span><i class="conum" data-value="4"></i><b>(4)</b></code></pre> |
|||
</div> |
|||
</div> |
|||
<div class="colist arabic"> |
|||
<table> |
|||
<tr> |
|||
<td><i class="conum" data-value="1"></i><b>1</b></td> |
|||
<td>create bucket</td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="2"></i><b>2</b></td> |
|||
<td>Apply policy</td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="3"></i><b>3</b></td> |
|||
<td>create user</td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="4"></i><b>4</b></td> |
|||
<td>Apply policy User2bucket</td> |
|||
</tr> |
|||
</table> |
|||
</div> |
|||
</div> |
|||
<div class="sect2"> |
|||
<h3 id="_sync_local2server">1.4. Sync local2Server</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">sync local2server</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"><span class="error">export MINIO_MIRROR_PARAMETERS="--delete"</span> |
|||
<span class="error">export MINIO_SERVER_URL="ip:9443" </span><i class="conum" data-value="1"></i><b>(1)</b> |
|||
<span class="error">export MINIO_PROT=https</span> |
|||
<span class="error">export MINIO_SERVER_DOCUMENTROOT="/home/user/syncdir" </span><i class="conum" data-value="2"></i><b>(2)</b> |
|||
<span class="error">export MINIO_WEB_BUCKET=test1</span> |
|||
<span class="error">export MINIO_WEB_PASS="bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd"</span> |
|||
<span class="error">export AWS_CONFIG_FILE="$HOME/.aws/config"</span> |
|||
<span class="error">export AWS_ACCESS_KEY_ID=$MINIO_WEB_BUCKET</span> |
|||
<span class="error">export AWS_SECRET_ACCESS_KEY=$MINIO_WEB_PASS</span> |
|||
|
|||
<span class="error">aws --endpoint-url $MINIO_PROT://$MINIO_SERVER_URL s3 sync MINIO_SERVER_DOCUMENTROOT s3://$MINIO_WEB_BUCKET $MINIO_MIRROR_PARAMETERS</span></code></pre> |
|||
</div> |
|||
</div> |
|||
<div class="colist arabic"> |
|||
<table> |
|||
<tr> |
|||
<td><i class="conum" data-value="1"></i><b>1</b></td> |
|||
<td>Minio Server ip</td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="2"></i><b>2</b></td> |
|||
<td>Directory sync</td> |
|||
</tr> |
|||
</table> |
|||
</div> |
|||
</div> |
|||
<div class="sect2"> |
|||
<h3 id="_sync_server2local">1.5. Sync Server2Local</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">sync server2local</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"><span class="error">export MINIO_MIRROR_PARAMETERS="--delete"</span> |
|||
<span class="error">export MINIO_SERVER_URL="ip:9443" </span><i class="conum" data-value="1"></i><b>(1)</b> |
|||
<span class="error">export MINIO_PROT=https</span> |
|||
<span class="error">export MINIO_SERVER_DOCUMENTROOT="/var/www/html" </span><i class="conum" data-value="2"></i><b>(2)</b> |
|||
<span class="error">export MINIO_WEB_BUCKET=test</span> |
|||
<span class="error">export MINIO_WEB_PASS="bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd"</span> |
|||
<span class="error">export AWS_CONFIG_FILE="$HOME/.aws/config"</span> |
|||
<span class="error">export AWS_ACCESS_KEY_ID=$MINIO_WEB_BUCKET</span> |
|||
<span class="error">export AWS_SECRET_ACCESS_KEY=$MINIO_WEB_PASS</span> |
|||
|
|||
<span class="error">aws --endpoint-url $MINIO_PROT://$MINIO_SERVER_URL s3 sync s3://$MINIO_WEB_BUCKET $MINIO_SERVER_DOCUMENTROOT $MINIO_MIRROR_PARAMETERS</span></code></pre> |
|||
</div> |
|||
</div> |
|||
<div class="colist arabic"> |
|||
<table> |
|||
<tr> |
|||
<td><i class="conum" data-value="1"></i><b>1</b></td> |
|||
<td>Minio Server ip</td> |
|||
</tr> |
|||
<tr> |
|||
<td><i class="conum" data-value="2"></i><b>2</b></td> |
|||
<td>Directory sync</td> |
|||
</tr> |
|||
</table> |
|||
</div> |
|||
</div> |
|||
<div class="sect2"> |
|||
<h3 id="_docker_compose_env">1.6. Docker-compose env</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">sync docker-compose env</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"> <span class="key">environment</span>: <i class="conum" data-value="1"></i><b>(1)</b> |
|||
- <span class="string"><span class="content">MINIO_MIRROR_PARAMETERS= --delete</span></span> |
|||
- <span class="string"><span class="content">MINIO_SERVER_URL=ip:9443</span></span> |
|||
- <span class="string"><span class="content">MINIO_PROT=https</span></span> |
|||
- <span class="string"><span class="content">MINIO_SERVER_DOCUMENTROOT=/var/www/html</span></span> |
|||
- <span class="string"><span class="content">MINIO_WEB_BUCKET=test</span></span> |
|||
- <span class="string"><span class="content">MINIO_WEB_PASS=bfksdkdjhdbhfbsdhbhf79fcbc7idjfdsjfsdbhfbhdbfhsd</span></span> |
|||
- <span class="string"><span class="content">AWS_CONFIG_FILE=/home/user/.aws/config</span></span></code></pre> |
|||
</div> |
|||
</div> |
|||
<div class="colist arabic"> |
|||
<table> |
|||
<tr> |
|||
<td><i class="conum" data-value="1"></i><b>1</b></td> |
|||
<td>add env to docker-comose</td> |
|||
</tr> |
|||
</table> |
|||
</div> |
|||
</div> |
|||
<div class="sect2"> |
|||
<h3 id="_automating_sync_with_cron">1.7. Automating sync with cron</h3> |
|||
<div class="listingblock"> |
|||
<div class="title">crond file</div> |
|||
<div class="content"> |
|||
<pre class="CodeRay highlight"><code data-lang="yaml"><span class="error">*/15 * * * * root /PATH/local2server >> /var/log/usercron.log 2>&1 / </span><i class="conum" data-value="1"></i><b>(1)</b></code></pre> |
|||
</div> |
|||
</div> |
|||
<div class="colist arabic"> |
|||
<table> |
|||
<tr> |
|||
<td><i class="conum" data-value="1"></i><b>1</b></td> |
|||
<td>Every 15 minutes</td> |
|||
</tr> |
|||
</table> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
</div> |
|||
<div id="footer"> |
|||
<div id="footer-text"> |
|||
Last updated 2021-04-05 18:27:18 UTC |
|||
</div> |
|||
</div> |
|||
<style> |
|||
/* Stylesheet for CodeRay to match GitHub theme | MIT License | http://foundation.zurb.com */ |
|||
pre.CodeRay{background:#f7f7f8} |
|||
.CodeRay .line-numbers{border-right:1px solid currentColor;opacity:.35;padding:0 .5em 0 0} |
|||
.CodeRay span.line-numbers{display:inline-block;margin-right:.75em} |
|||
.CodeRay .line-numbers strong{color:#000} |
|||
table.CodeRay{border-collapse:separate;border:0;margin-bottom:0;background:none} |
|||
table.CodeRay td{vertical-align:top;line-height:inherit} |
|||
table.CodeRay td.line-numbers{text-align:right} |
|||
table.CodeRay td.code{padding:0 0 0 .75em} |
|||
.CodeRay .debug{color:#fff !important;background:#000080 !important} |
|||
.CodeRay .annotation{color:#007} |
|||
.CodeRay .attribute-name{color:#000080} |
|||
.CodeRay .attribute-value{color:#700} |
|||
.CodeRay .binary{color:#509} |
|||
.CodeRay .comment{color:#998;font-style:italic} |
|||
.CodeRay .char{color:#04d} |
|||
.CodeRay .char .content{color:#04d} |
|||
.CodeRay .char .delimiter{color:#039} |
|||
.CodeRay .class{color:#458;font-weight:bold} |
|||
.CodeRay .complex{color:#a08} |
|||
.CodeRay .constant,.CodeRay .predefined-constant{color:#008080} |
|||
.CodeRay .color{color:#099} |
|||
.CodeRay .class-variable{color:#369} |
|||
.CodeRay .decorator{color:#b0b} |
|||
.CodeRay .definition{color:#099} |
|||
.CodeRay .delimiter{color:#000} |
|||
.CodeRay .doc{color:#970} |
|||
.CodeRay .doctype{color:#34b} |
|||
.CodeRay .doc-string{color:#d42} |
|||
.CodeRay .escape{color:#666} |
|||
.CodeRay .entity{color:#800} |
|||
.CodeRay .error{color:#808} |
|||
.CodeRay .exception{color:inherit} |
|||
.CodeRay .filename{color:#099} |
|||
.CodeRay .function{color:#900;font-weight:bold} |
|||
.CodeRay .global-variable{color:#008080} |
|||
.CodeRay .hex{color:#058} |
|||
.CodeRay .integer,.CodeRay .float{color:#099} |
|||
.CodeRay .include{color:#555} |
|||
.CodeRay .inline{color:#000} |
|||
.CodeRay .inline .inline{background:#ccc} |
|||
.CodeRay .inline .inline .inline{background:#bbb} |
|||
.CodeRay .inline .inline-delimiter{color:#d14} |
|||
.CodeRay .inline-delimiter{color:#d14} |
|||
.CodeRay .important{color:#555;font-weight:bold} |
|||
.CodeRay .interpreted{color:#b2b} |
|||
.CodeRay .instance-variable{color:#008080} |
|||
.CodeRay .label{color:#970} |
|||
.CodeRay .local-variable{color:#963} |
|||
.CodeRay .octal{color:#40e} |
|||
.CodeRay .predefined{color:#369} |
|||
.CodeRay .preprocessor{color:#579} |
|||
.CodeRay .pseudo-class{color:#555} |
|||
.CodeRay .directive{font-weight:bold} |
|||
.CodeRay .type{font-weight:bold} |
|||
.CodeRay .predefined-type{color:inherit} |
|||
.CodeRay .reserved,.CodeRay .keyword {color:#000;font-weight:bold} |
|||
.CodeRay .key{color:#808} |
|||
.CodeRay .key .delimiter{color:#606} |
|||
.CodeRay .key .char{color:#80f} |
|||
.CodeRay .value{color:#088} |
|||
.CodeRay .regexp .delimiter{color:#808} |
|||
.CodeRay .regexp .content{color:#808} |
|||
.CodeRay .regexp .modifier{color:#808} |
|||
.CodeRay .regexp .char{color:#d14} |
|||
.CodeRay .regexp .function{color:#404;font-weight:bold} |
|||
.CodeRay .string{color:#d20} |
|||
.CodeRay .string .string .string{background:#ffd0d0} |
|||
.CodeRay .string .content{color:#d14} |
|||
.CodeRay .string .char{color:#d14} |
|||
.CodeRay .string .delimiter{color:#d14} |
|||
.CodeRay .shell{color:#d14} |
|||
.CodeRay .shell .delimiter{color:#d14} |
|||
.CodeRay .symbol{color:#990073} |
|||
.CodeRay .symbol .content{color:#a60} |
|||
.CodeRay .symbol .delimiter{color:#630} |
|||
.CodeRay .tag{color:#008080} |
|||
.CodeRay .tag-special{color:#d70} |
|||
.CodeRay .variable{color:#036} |
|||
.CodeRay .insert{background:#afa} |
|||
.CodeRay .delete{background:#faa} |
|||
.CodeRay .change{color:#aaf;background:#007} |
|||
.CodeRay .head{color:#f8f;background:#505} |
|||
.CodeRay .insert .insert{color:#080} |
|||
.CodeRay .delete .delete{color:#800} |
|||
.CodeRay .change .change{color:#66f} |
|||
.CodeRay .head .head{color:#f4f} |
|||
</style> |
|||
</body> |
|||
</html> |
|||
File diff suppressed because it is too large
@ -0,0 +1,406 @@ |
|||
= Swarm Storage HowTo! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
:description: IoT Εισαγωγή στο Cloud |
|||
:keywords: Cloud, swarm |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Πίνακας περιεχομένων |
|||
:toclevels: 4 |
|||
:source-highlighter: coderay |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
{empty} + |
|||
|
|||
== Install Minio |
|||
|
|||
=== Create Docker secrets for MinIO |
|||
|
|||
.create secrets |
|||
[source,sh] |
|||
---- |
|||
KEY=$(od -vN 32 -An -tx1 /dev/urandom | tr -d ' \n' ; echo) |
|||
SECRET=$(od -vN 32 -An -tx1 /dev/urandom | tr -d ' \n' ; echo) |
|||
echo $KEY > key |
|||
echo $SECRET > secret |
|||
echo $KEY | docker secret create access_key - |
|||
echo $SECRET | docker secret create secret_key - |
|||
---- |
|||
|
|||
|
|||
|
|||
=== Create node labels |
|||
|
|||
.create labels |
|||
[source,sh] |
|||
---- |
|||
docker node update --label-add minio1=true [node-name] // <1> |
|||
docker node update --label-add minio2=true [node-name] |
|||
docker node update --label-add minio3=true [node-name] |
|||
docker node update --label-add minio4=true [node-name] |
|||
|
|||
docker node update --label-add group=minio [node-name] //<2> |
|||
docker node update --label-add group=minio [node-name] |
|||
---- |
|||
<1> node name from command: docker node ls e.g. *snf-12118* (minio) |
|||
<2> node name from command: docker node ls e.g. *snf-12118* (proxy) |
|||
|
|||
|
|||
=== Generate a Certificate |
|||
|
|||
.Create a configuration file (openssl.conf) |
|||
[source,sh] |
|||
---- |
|||
[req] |
|||
distinguished_name = req_distinguished_name |
|||
x509_extensions = v3_req |
|||
prompt = no |
|||
|
|||
[req_distinguished_name] |
|||
C = US //<1> |
|||
ST = VA //<1> |
|||
L = Somewhere //<1> |
|||
O = MyOrg //<1> |
|||
OU = MyOU //<1> |
|||
CN = MyServerName //<1> |
|||
|
|||
[v3_req] |
|||
subjectAltName = @alt_names |
|||
|
|||
[alt_names] |
|||
IP.1 = 127.0.0.1 //<2> |
|||
---- |
|||
<1> change to the correct values |
|||
<2> change to the correct IP address |
|||
|
|||
.Run openssl and specify the configuration file |
|||
[source,sh] |
|||
---- |
|||
openssl req -x509 -nodes -days 730 -newkey rsa:2048 -keyout private.key -out public.crt -config openssl.conf |
|||
---- |
|||
|
|||
|
|||
=== Create Yaml file |
|||
|
|||
.docker-compose |
|||
[source,yaml] |
|||
---- |
|||
services: |
|||
minio1: //<1> |
|||
image: minio/minio:RELEASE.2020-04-10T03-34-42Z //<2> |
|||
hostname: minio1 |
|||
volumes: |
|||
- minio1-data:/export //<3> |
|||
ports: |
|||
- "9001:9000" //<4> |
|||
networks: |
|||
- minio_distributed //<5> |
|||
deploy: |
|||
restart_policy: |
|||
delay: 10s |
|||
max_attempts: 10 |
|||
window: 60s |
|||
placement: |
|||
constraints: |
|||
- node.labels.minio1==true //<6> |
|||
command: server http://minio{1...4}/export //<7> |
|||
secrets: //<8> |
|||
- secret_key |
|||
- access_key |
|||
healthcheck: |
|||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] //<9> |
|||
interval: 30s |
|||
timeout: 20s |
|||
retries: 3 |
|||
|
|||
minio2: //<10> |
|||
image: minio/minio:RELEASE.2020-04-10T03-34-42Z |
|||
hostname: minio2 //<10> |
|||
volumes: |
|||
- minio2-data:/export //<11> |
|||
ports: |
|||
- "9002:9000" //<12> |
|||
networks: |
|||
- minio_distributed //<5> |
|||
deploy: |
|||
restart_policy: |
|||
delay: 10s |
|||
max_attempts: 10 |
|||
window: 60s |
|||
placement: |
|||
constraints: |
|||
- node.labels.minio2==true //<13> |
|||
command: server http://minio{1...4}/export |
|||
secrets: |
|||
- secret_key |
|||
- access_key |
|||
healthcheck: |
|||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] |
|||
interval: 30s |
|||
timeout: 20s |
|||
retries: 3 |
|||
|
|||
minio3: |
|||
image: minio/minio:RELEASE.2020-04-10T03-34-42Z |
|||
hostname: minio3 |
|||
volumes: |
|||
- minio3-data:/export |
|||
ports: |
|||
- "9003:9000" |
|||
networks: |
|||
- minio_distributed //<5> |
|||
deploy: |
|||
restart_policy: |
|||
delay: 10s |
|||
max_attempts: 10 |
|||
window: 60s |
|||
placement: |
|||
constraints: |
|||
- node.labels.minio3==true |
|||
command: server http://minio{1...4}/export |
|||
secrets: |
|||
- secret_key |
|||
- access_key |
|||
healthcheck: |
|||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] |
|||
interval: 30s |
|||
timeout: 20s |
|||
retries: 3 |
|||
|
|||
minio4: |
|||
image: minio/minio:RELEASE.2020-04-10T03-34-42Z |
|||
hostname: minio4 |
|||
volumes: |
|||
- minio4-data:/export |
|||
ports: |
|||
- "9004:9000" |
|||
networks: |
|||
- minio_distributed //<5> |
|||
deploy: |
|||
restart_policy: |
|||
delay: 10s |
|||
max_attempts: 10 |
|||
window: 60s |
|||
placement: |
|||
constraints: |
|||
- node.labels.minio4==true |
|||
command: server http://minio{1...4}/export |
|||
secrets: |
|||
- secret_key |
|||
- access_key |
|||
healthcheck: |
|||
test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"] |
|||
interval: 30s |
|||
timeout: 20s |
|||
retries: 3 |
|||
|
|||
web: |
|||
image: nginx:1.17.9-alpine |
|||
deploy: |
|||
mode: replicated |
|||
restart_policy: |
|||
delay: 10s |
|||
max_attempts: 10 |
|||
window: 60s |
|||
replicas: 2 |
|||
placement: |
|||
max_replicas_per_node: 1 |
|||
constraints: |
|||
- node.labels.group==minio //<14> |
|||
ports: |
|||
- "8080:80" |
|||
- "9443:443" |
|||
volumes: //<15> |
|||
- /PATH_to_FILE/minio.conf:/etc/nginx/conf.d/default.conf //<16> |
|||
- /PATH_to_FILE/public.crt:/etc/nginx/public.crt //<17> |
|||
- /PATH_to_FILE/private.key:/etc/nginx/private.key //<17> |
|||
networks: |
|||
- minio_distributed //<5> |
|||
|
|||
|
|||
volumes: |
|||
minio1-data: |
|||
|
|||
minio2-data: |
|||
|
|||
minio3-data: |
|||
|
|||
minio4-data: |
|||
|
|||
|
|||
networks: |
|||
minio_distributed: //<5> |
|||
driver: overlay |
|||
|
|||
secrets: |
|||
secret_key: |
|||
external: true |
|||
access_key: |
|||
external: true |
|||
|
|||
---- |
|||
<1> Service name |
|||
<2> Image name |
|||
<3> Volume to Use |
|||
<4> Expose port |
|||
<5> Network to Use |
|||
<6> Node Placement |
|||
<7> Start server |
|||
<8> insert secrets |
|||
<9> health check command |
|||
<10> *NEW* Service name |
|||
<11> *NEW* Volume |
|||
<12> *NEW* Port |
|||
<13> *NEW* Label |
|||
<14> Node Placement (Proxy) |
|||
<15> Bind mount config files |
|||
<16> Nginx config file |
|||
<17> ssl keys |
|||
|
|||
|
|||
|
|||
=== Create config file (proxy) |
|||
|
|||
.nginx config |
|||
[source,yaml] |
|||
---- |
|||
upstream minio_servers { |
|||
server minio1:9000; //<1> |
|||
server minio2:9000; //<1> |
|||
server minio3:9000; //<1> |
|||
server minio4:9000; //<1> |
|||
} |
|||
proxy_cache_path /var/tmp levels=1:2 keys_zone=my_cache:10m max_size=10g inactive=60m use_temp_path=off; |
|||
server { |
|||
listen 80; |
|||
server_name name.example.org; //<2> |
|||
return 301 https://name.example.org$request_uri; // <3> |
|||
} |
|||
server { |
|||
listen 443 ssl; |
|||
server_name name.example.org; |
|||
|
|||
# To allow special characters in headers |
|||
ignore_invalid_headers off; |
|||
# Allow any size file to be uploaded. |
|||
# Set to a value such as 1000m; to restrict file size to a specific value |
|||
client_max_body_size 0; |
|||
# To disable buffering |
|||
proxy_buffering off; |
|||
|
|||
ssl_certificate /etc/nginx/public.crt; //<4> |
|||
ssl_certificate_key /etc/nginx/private.key; //<4> |
|||
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; |
|||
ssl_ciphers HIGH:!aNULL:!MD5; |
|||
|
|||
location / { |
|||
proxy_cache my_cache; |
|||
proxy_set_header X-Real-IP $remote_addr; |
|||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; |
|||
proxy_set_header X-Forwarded-Proto $scheme; |
|||
proxy_set_header Host $http_host; |
|||
|
|||
proxy_set_header X-NginX-Proxy true; |
|||
proxy_ssl_session_reuse off; |
|||
proxy_redirect off; |
|||
|
|||
|
|||
proxy_connect_timeout 300; |
|||
# Default is HTTP/1, keepalive is only enabled in HTTP/1.1 |
|||
proxy_http_version 1.1; |
|||
proxy_set_header Connection ""; |
|||
chunked_transfer_encoding off; |
|||
|
|||
#proxy_pass http://minio1:9000; # If you are using docker-compose this would be the hostname i.e. minio |
|||
proxy_pass http://minio_servers; // <5> |
|||
# Health Check endpoint might go here. See https://www.nginx.com/resources/wiki/modules/healthcheck/ |
|||
# /minio/health/live; |
|||
} |
|||
} |
|||
|
|||
---- |
|||
<1> Service names from yaml |
|||
<2> Server name or IP |
|||
<3> Redirect to https |
|||
<4> keys |
|||
<5> pass to servers |
|||
|
|||
=== Copy files to nodes |
|||
|
|||
.cp files |
|||
[source,yaml] |
|||
---- |
|||
# copy files to proxy server |
|||
scp minio.conf user@IP:/PATH_to_FILE/minio.conf // <1> |
|||
scp private.key user@IP:/PATH_to_FILE/private.key // <1> |
|||
scp public.crt user@IP:/PATH_to_FILE/public.crt // <1> |
|||
---- |
|||
<1> change *ip* (see <2> in http://docs.swarmlab.io/lab/DockerSwarm/swarm-volumes-storage-howto.adoc.html#_create_node_labels[create_node_labels]) and *PATH_to_FILE* (see <16> in http://docs.swarmlab.io/lab/DockerSwarm/swarm-volumes-storage-howto.adoc.html#_create_yaml_file[create_yaml_file]) |
|||
|
|||
|
|||
=== deploy |
|||
|
|||
.stack deploy |
|||
[source,yaml] |
|||
---- |
|||
docker stack deploy --compose-file=docker-compose.yaml minio_stack |
|||
---- |
|||
|
|||
=== Test MinIO in Browser |
|||
|
|||
Point your web browser to http://ip:9443 |
|||
|
|||
image:./minio-browser.png[] |
|||
|
|||
== Install tools |
|||
|
|||
|
|||
=== Install AWS CLI |
|||
|
|||
Universal Command Line Interface for Amazon Web Services |
|||
|
|||
.aws cli |
|||
[source,sh] |
|||
---- |
|||
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip" |
|||
unzip awscliv2.zip |
|||
sudo ./aws/install |
|||
---- |
|||
|
|||
The AWS Command Line Interface (AWS CLI) is an open source tool that enables you to interact with AWS services using commands in your command-line shell |
|||
|
|||
.create file /home/user/.aws/credentials |
|||
[source,sh] |
|||
---- |
|||
[default] |
|||
aws_secret_access_key = key // <1> |
|||
aws_access_key_id = secret // <1> |
|||
---- |
|||
<1> http://docs.swarmlab.io/lab/DockerSwarm/swarm-volumes-minio-howto.adoc.html#_apply_policy[see^] |
|||
|
|||
|
|||
|
|||
.create file /home/user/.aws/config |
|||
[source,sh] |
|||
---- |
|||
[default] |
|||
s3 = |
|||
signature_version = s3v4 |
|||
region = us-east-1 |
|||
---- |
|||
=== Install mc client |
|||
|
|||
MinIO Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4). |
|||
|
|||
.mc |
|||
[source,yaml] |
|||
---- |
|||
wget https://dl.min.io/client/mc/release/linux-amd64/mc |
|||
chmod +x mc |
|||
./mc --help |
|||
---- |
|||
|
|||
|
|||
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
@ -0,0 +1,219 @@ |
|||
= Swarm Storage ! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
:description: IoT Εισαγωγή στο Cloud |
|||
:keywords: Cloud, swarm |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Πίνακας περιεχομένων |
|||
:toclevels: 4 |
|||
:source-highlighter: coderay |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
{empty} + |
|||
|
|||
== Manage data |
|||
|
|||
By default all files created inside a container are stored on a writable container layer. This means that: |
|||
|
|||
- The data doesn’t persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it. |
|||
- A container’s writable layer is tightly coupled to the host machine where the container is running. You can’t easily move the data somewhere else. |
|||
- Writing into a container’s writable layer requires a storage driver to manage the filesystem. The storage driver provides a union filesystem, using the Linux kernel. This extra abstraction reduces performance as compared to using data volumes, which write directly to the host filesystem. |
|||
|
|||
|
|||
.Volumes and bind |
|||
[NOTE] |
|||
**** |
|||
Docker has two options for containers to store files in the host machine, so that the files are persisted even after the container stops: |
|||
|
|||
- volumes, |
|||
- and bind mounts. |
|||
|
|||
On Linux you can also use a tmpfs mount. |
|||
**** |
|||
image:./types-of-mounts-volume.png[alt="Volumes"] + |
|||
|
|||
No matter which type of mount you choose to use, the data looks the same from within the container. It is exposed as either a directory or an individual file in the container’s filesystem. |
|||
|
|||
|
|||
|
|||
* Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux). |
|||
** Non-Docker processes should not modify this part of the filesystem. |
|||
|
|||
* Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. |
|||
** Non-Docker processes on the Docker host or a Docker container can modify them at any time. |
|||
|
|||
* tmpfs mounts are stored in the host system’s memory only, and are never written to the host system’s filesystem. |
|||
|
|||
|
|||
=== Volumes |
|||
|
|||
Created and managed by Docker. You can create a volume explicitly using the *docker volume create* command, or Docker can create a volume during container or service creation. |
|||
|
|||
When you create a volume, it is stored within a directory on the Docker host. When you mount the volume into a container, this directory is what is mounted into the container. This is similar to the way that bind mounts work, except that volumes are managed by Docker and are isolated from the core functionality of the host machine. |
|||
|
|||
A given volume *can be mounted into multiple containers* simultaneously. When no running container is using a volume, the volume is still available to Docker and is not removed automatically. You can remove unused volumes using *docker volume prune.* |
|||
|
|||
Volumes also support the use of volume drivers, which allow you to store your data on remote hosts or cloud providers, among other possibilities. |
|||
|
|||
|
|||
==== use cases for volumes |
|||
|
|||
|
|||
- *Sharing data among multiple running containers.* If you don’t explicitly create it, a volume is created the first time it is mounted into a container. When that container stops or is removed, the volume still exists. Multiple containers can mount the same volume simultaneously, either read-write or read-only. Volumes are only removed when you explicitly remove them. |
|||
|
|||
- When the Docker host is not guaranteed to have a given directory or file structure. Volumes help you decouple the configuration of the Docker host from the container runtime. |
|||
|
|||
- When you want to *store your container’s data on a remote host or a cloud provider,* rather than locally. |
|||
|
|||
- When you need to back up, restore, or migrate data from one Docker host to another, volumes are a better choice. You can stop containers using the volume, then back up the volume’s directory (such as /var/lib/docker/volumes/<volume-name>). |
|||
|
|||
|
|||
|
|||
|
|||
=== Bind mounts |
|||
|
|||
Bind mounts have limited functionality compared to volumes. |
|||
|
|||
When you use a bind mount, a *file or directory on the host machine* is mounted into a container. The file or directory is referenced by its full path on the host machine. |
|||
|
|||
The file or directory does not need to exist on the Docker host already. It is *created on demand* if it does not yet exist. |
|||
|
|||
*Bind mounts are very performant, but they rely on the host machine’s filesystem* having a specific directory structure available. |
|||
|
|||
.Bind mounts allow access to sensitive files |
|||
[NOTE] |
|||
==== |
|||
One side effect of using bind mounts, for better or for worse, is that *you can change the host filesystem via processes running in a container,* including creating, modifying, or deleting important system files or directories. |
|||
This is a powerful ability which can have security implications, including impacting non-Docker processes on the host system |
|||
==== |
|||
|
|||
|
|||
|
|||
==== use cases for bind mounts |
|||
|
|||
|
|||
|
|||
- *Sharing configuration files* from the host machine to containers. This is how Docker provides DNS resolution to containers by default, by mounting /etc/resolv.conf from the host machine into each container. |
|||
|
|||
- *Sharing source code* or build artifacts between a development environment on the Docker host and a container. For instance, you may mount a Project target/ directory into a container, and each time you build the Project on the Docker host, the container gets access to the rebuilt artifacts. |
|||
|
|||
|
|||
- When the file or directory structure of the Docker host is guaranteed to be consistent with the bind mounts the containers require. |
|||
|
|||
|
|||
|
|||
=== tmpfs mounts |
|||
|
|||
A tmpfs mount is not persisted on disk, either on the Docker host or within a container. It can be used by a container during the lifetime of the container, to store non-persistent state or sensitive information. For instance, internally, swarm services use tmpfs mounts to mount secrets into a service’s containers. |
|||
|
|||
|
|||
==== use cases for tmpfs mount |
|||
|
|||
- tmpfs mounts are best used for cases when *you do not want the data to persist* either on the host machine or within the container. This may be *for security reasons or to protect the performance* of the container when your application needs to write a large volume of non-persistent state data. |
|||
|
|||
|
|||
.keep the following in mind |
|||
[NOTE] |
|||
==== |
|||
- If you mount an *empty volume* into a directory in the container in which files or directories exist, these files or directories are propagated (copied) into the volume. Similarly, if you start a container and specify a volume which does not already exist, an empty volume is created for you. This is a good way to pre-populate data that another container needs. |
|||
|
|||
- If you mount a *bind mount or non-empty volume* into a directory in the container in which some files or directories exist, these files or directories are obscured by the mount, just as if you saved files into /mnt on a Linux host and then mounted a USB drive into /mnt. The contents of /mnt would be obscured by the contents of the USB drive until the USB drive were unmounted. The obscured files are not removed or altered, but are not accessible while the bind mount or volume is mounted. |
|||
==== |
|||
|
|||
== Share data among machines |
|||
|
|||
When building fault-tolerant applications, you need to configure multiple replicas of the same service to have access to the same files. |
|||
|
|||
image:./volumes-shared-storage.svg[alt="Shared data"] + |
|||
|
|||
There are several ways to achieve this when developing your applications. |
|||
|
|||
- One is to *add logic to your application* to store files on a cloud object storage system like *Amazon S3.* |
|||
- Another is to *create volumes* with a driver that supports writing files to an *external storage* system like *NFS* or *Amazon S3.* |
|||
|
|||
.Volume drivers |
|||
[NOTE] |
|||
==== |
|||
Volume drivers allow you to abstract the underlying storage system from the application logic. |
|||
==== |
|||
|
|||
=== External storage system |
|||
There are a lot of different systems to use. |
|||
|
|||
We are going to look at the two most poppular systems (they both support any combination of the aforementioned ways). |
|||
|
|||
A more detailed list of systems can be found at the bottom of the page. |
|||
|
|||
==== MINIO |
|||
|
|||
|
|||
https://en.wikipedia.org/wiki/MinIO[High Performance Object Storage,^] |
|||
|
|||
|
|||
.open source |
|||
[NOTE] |
|||
**** |
|||
MinIO is 100% *open source under the Apache V2 license.* This means that MinIO’s customers are free from lock in, free to inspect, free to innovate, free to modify and free to redistribute. |
|||
**** |
|||
|
|||
|
|||
.Scaling |
|||
[NOTE] |
|||
**** |
|||
At MinIO, scaling starts with a single cluster which can be federated with other MinIO clusters to create a global namespace, spanning multiple data centers if needed. |
|||
**** |
|||
|
|||
image:./scalability.svg[alt="scale"] |
|||
|
|||
|
|||
|
|||
|
|||
.S3 compatibility |
|||
[NOTE] |
|||
**** |
|||
Amazon’s S3 API is the defacto standard in the object storage world. MinIO is the defacto standard for S3 compatibility and was one of the first to adopt the API and the first to add support for S3 Select. |
|||
**** |
|||
|
|||
image:./s3-compatibility.svg[alt="s3"] |
|||
|
|||
|
|||
==== IPFS (Inter-Planetary File System) |
|||
|
|||
A peer-to-peer hypermedia https://en.wikipedia.org/wiki/InterPlanetary_File_System[protocol,^] designed to make the web faster, safer, and more open. |
|||
|
|||
.distributed system |
|||
[NOTE] |
|||
**** |
|||
IPFS is a distributed system for storing and accessing files, websites, applications, and data. |
|||
**** |
|||
|
|||
.IPFS Cluster |
|||
[NOTE] |
|||
**** |
|||
|
|||
Automated data availability and redundancy on IPFS |
|||
|
|||
IPFS Cluster provides data orchestration across a swarm of IPFS daemons by allocating, replicating and tracking a global pinset distributed among multiple peers. |
|||
**** |
|||
|
|||
image:./cluster.png[] |
|||
|
|||
|
|||
|
|||
|
|||
|
|||
|
|||
=== Extend docker |
|||
|
|||
You can extend the capabilities of the Docker Engine by loading third-party plugins. |
|||
|
|||
Docker supports *authorization,* *volume* and *network* driver plugins |
|||
|
|||
|
|||
|
|||
==== available plugins |
|||
|
|||
https://hub.docker.com/search?q=&type=plugin[^] |
|||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
|
After Width: | Height: | Size: 80 KiB |
|
After Width: | Height: | Size: 23 KiB |
|
After Width: | Height: | Size: 9.1 KiB |
Loading…
Reference in new issue