= List!
image::ROOT:swarmlab.png[swarmlab,150,float=right]
== Why Docker
https://www.cloudbees.com/blog/why-dockero[Why Docker^]
== Intro docker
https://www.compose-spec.io/[docker-compose^]
https://github.com/veggiemonk/awesome-docker[github docker-compose^]
https://twitter.com/awesome_docker[github awesome docker]
=== Learn to build and deploy your distributed applications easily to the cloud with Docker
https://docker-curriculum.com/[HowTo]
== Next Step DockerSwarm
****
TIP: DockerSwarm
video::Tm0Q5zr3FL4[youtube, start=0]
****
https://www.simplilearn.com/tutorials/docker-tutorial/docker-swarm[docker-swarm tutorial^]
=== Getting started with swarm - official from docker
https://docs.docker.com/engine/swarm/swarm-tutorial/https://k21academy.com/docker-kubernetes/docker-swarm/[docker-swarm^]
=== Replicability, use the same files as when developing locally.
https://dockerswarm.rocks/[dockerswarm.rocks^]
=== Open Source Cloud Guide από την IBM
https://open-cloud-guide.dev/[open-cloud huide^]
=== Alternatives
https://www.nomadproject.io/[monad^]
https://dev.to/stevenmcgown/kubernetes-for-dummies-5hmh[kubernetes^]
=== Raft consensus in swarm mode
https://docs.docker.com/engine/swarm/raft/[raft^]
https://raft.github.io/[raft github^]
http://thesecretlivesofdata.com/raft/[raft demo^]
== Next Step Storage
****
TIP: Storage
video::W2Z7fbCLSTw[youtube, start=0]
****
=== Generic
https://docs.mongodb.com/manual/replication/[Mongo replica^]
https://docs.mongodb.com/manual/core/capped-collections/[mongo capped^]
=== On memory - Key-value
https://redis.io/topics/cluster-tutorial[Redis cluster^]
https://redis.io/[redis server^]
https://www.instaclustr.com/blog/redis-vs-memcached/[redis vs memcached^]
https://etcd.io/[etcd^]
****
TIP: etcd
video::OmphHSaO1sE[youtube, start=0]
****
****
TIP: redis
video::G1rOthIU-uo[youtube, start=0]
****
*etcd vs. Redis*
Like etcd, Redis is an open source tool, but their basic functionalities are different.
[cols="1,1"]
|===
|Redis|etcd
| Redis is an in-memory data store and can function as a database, cache, or message broker. Redis supports a wider variety of data types and structures than etcd and has much faster read/write performance.
| etcd has superior fault tolerance, stronger failover and continuous data availability capabilities.
|Redis is better suited for serving as a distributed memory caching system than for storing and distributed system configuration information.
|etcd persists all stored data to disk.
|===
=== Sharing
==== NFS
https://cloud.netapp.com/blog/azure-anf-blg-linux-nfs-server-how-to-set-up-server-and-client[NFS^]
https://www.gluster.org/[GlusterFS^]
== Deploy
https://www.compose-spec.io/[docker-compose^]
****
TIP: Deploy
video::zLltQxVb3NI[youtube, start=0]
****
https://12factor.net/[12factor^]
== Development - From Design to Application
=== programming language
==== nodeJS
****
TIP: NodeJS
video::ENrzD9HAZK4[youtube, start=0]
****
https://nodejs.org/en/[nodeJS^]
https://p5js.org/[p5js^]
https://threejs.org/[threejs^]
==== python
https://www.python.org/[python^]
=== Big data and Scientific analysis
https://numpy.org/[numpy^]
https://scipy.org/[scipy^]
https://jupyter.org/[jupyter^]
=== Rest API
****
TIP: Rest api
video::s7wmiS2mSXY[youtube, start=0]
****
=== Sockets
****
TIP: Sockets
video::ZBM28ZPlin8[youtube, start=0]
****
https://socket.io/[socket.io^]
=== Swarm intelligence
==== TensorSwarm: A framework for reinforcement learning of robot swarms.
https://github.com/TensorSwarm/TensorSwarm[TensorSwarm^]
==== ROS - Robot Operating System
https://www.ros.org/[ROS]
==== Reinforcement Learning
https://www.tensorflow.org/agents/tutorials/0_intro_rl[Introduction to RL^]
https://www.tensorflow.org/js/guide/nodejs?hl=es[tensorflow.js^]
https://github.com/karpathy/reinforcejs[common RL algorithms^]
https://developer.ibm.com/tutorials/an-introduction-to-ai-in-nodejs/[An introduction to AI in Node.js^]
https://pytorch.org/[pytorch^]
https://reagent.ai/[pytorch - Reinforcement Learning Platform^]
*PyTorch vs. TensorFlow*
[cols="1,1"]
|===
|PyTorch | TensorFlow
|PyTorch is open source deep learning framework created by developers at Facebook and released in 2017.
|TensorFlow is open source deep learning framework created by developers at Google and released in 2015.
|*Top PyTorch Projects*
* CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. https://stanfordmlgroup.github.io/projects/chexnet/[url^]
* PYRO: Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend. https://pyro.ai/[url^]
* Horizon: A platform for applied reinforcement learning (Applied RL) https://horizonrl.com[url^]
|*Top TensorFlow Projects*
* Magenta: An open source research project exploring the role of machine learning as a tool in the creative process. https://magenta.tensorflow.org/[url^]
* Sonnet: Sonnet is a library built on top of TensorFlow for building complex neural networks. https://sonnet.dev/[url^]
* Ludwig: Ludwig is a toolbox to train and test deep learning models without the need to write code. https://uber.github.io/ludwig/[url^]
| *Production Deployment*
PyTorch doesn't provide any framework to deploy models directly on to the web. You'll have to use either Flask or Django as the backend server.
| *Production Deployment*
When it comes to deploying trained models to production, TensorFlow is the clear winner. We can directly deploy models in TensorFlow using TensorFlow serving which is a framework that uses REST Client API.
|*Visualization*
PyTorch developers use Visdom, the features provided by Visdom are very minimalistic and limited.
|*Visualization*
When it comes to visualization of the training process, TensorFlow takes the lead. Visualization helps the developer track the training process and debug in a more convenient way.
TenforFlow’s visualization library is called TensorBoard.
|Python
|Tensorflow has APIs for various langages :
* C (used to develop other APIs in practice, though)
* C++ (Tensorflow is written in C++ and CUDA)
* Java
* GO
* Rust
* Haskell (yes, even Haskell)
* C#
* Julia
* R
* Python
* And, of course, for JavaScript, since 2019.
|*Pytorch Cons:*
* Third-party needed for visualization.
* API server needed for production.
|*TensorFlow Cons:*
* Debugging method.
* Hard to make quick changes.
|===
== Auth
https://oauth.net/2/[auth2^]
https://oauth.net/code/nodejs/[Nodejs auth library^]
=== Auth server
https://www.keycloak.org/[auth2 server^]
== Security VPN etc
https://www.wireguard.com/[simply the best VPN out there^]
== POC
=== netflix
****
TIP: Netflix
video::CZ3wIuvmHeM[youtube, start=0]
****
6:50 monolithic
8:24 minroservices
19:00 microservice
see also 37:00
https://netflix.github.io/[netflix github^]
=== Robots and IoT
https://nodebots.io/[nodebot^]
http://johnny-five.io/[johny-five^]
https://ardupilot.org/[eduplot^]
https://github.com/ArduPilot/ardupilot[ardupilot^]
