Documentation/tech-list/modules/ROOT/pages/index.adoc

= List!
  
image::ROOT:swarmlab.png[swarmlab,150,float=right]


== Why Docker

https://www.cloudbees.com/blog/why-dockero[Why Docker^]

== Intro docker

https://www.compose-spec.io/[docker-compose^]

https://github.com/veggiemonk/awesome-docker[github docker-compose^]

https://twitter.com/awesome_docker[github awesome docker]

=== Learn to build and deploy your distributed applications easily to the cloud with Docker 

https://docker-curriculum.com/[HowTo]

== Next Step DockerSwarm

****
TIP: DockerSwarm

video::Tm0Q5zr3FL4[youtube, start=0]
****

https://www.simplilearn.com/tutorials/docker-tutorial/docker-swarm[docker-swarm tutorial^]

=== Getting started with swarm - official from docker

https://docs.docker.com/engine/swarm/swarm-tutorial/https://k21academy.com/docker-kubernetes/docker-swarm/[docker-swarm^]

=== Replicability, use the same files as when developing locally.

https://dockerswarm.rocks/[dockerswarm.rocks^]

=== Open Source Cloud Guide από την IBM

https://open-cloud-guide.dev/[open-cloud huide^]

=== Alternatives 

https://www.nomadproject.io/[monad^]

https://dev.to/stevenmcgown/kubernetes-for-dummies-5hmh[kubernetes^]

=== Raft consensus in swarm mode

https://docs.docker.com/engine/swarm/raft/[raft^]

https://raft.github.io/[raft github^]

http://thesecretlivesofdata.com/raft/[raft demo^]

== Next Step Storage

****
TIP: Storage

video::W2Z7fbCLSTw[youtube, start=0]
****

=== Generic

https://docs.mongodb.com/manual/replication/[Mongo replica^]

https://docs.mongodb.com/manual/core/capped-collections/[mongo capped^]

=== On memory - Key-value

https://redis.io/topics/cluster-tutorial[Redis cluster^]

https://redis.io/[redis server^]

https://www.instaclustr.com/blog/redis-vs-memcached/[redis vs memcached^]

https://etcd.io/[etcd^]

****
TIP: etcd

video::OmphHSaO1sE[youtube, start=0]
****

****
TIP: redis

video::G1rOthIU-uo[youtube, start=0]
****

*etcd vs. Redis*

Like etcd, Redis is an open source tool, but their basic functionalities are different.

[cols="1,1"]
|===
|Redis|etcd

| Redis is an in-memory data store and can function as a database, cache, or message broker. Redis supports a wider variety of data types and structures than etcd and has much faster read/write performance.
| etcd has superior fault tolerance, stronger failover and continuous data availability capabilities. 

|Redis is better suited for serving as a distributed memory caching system than for storing and distributed system configuration information.
|etcd persists all stored data to disk. 


|===





=== Sharing

==== NFS	

https://cloud.netapp.com/blog/azure-anf-blg-linux-nfs-server-how-to-set-up-server-and-client[NFS^]

https://www.gluster.org/[GlusterFS^]	   


== Deploy

https://www.compose-spec.io/[docker-compose^]

****
TIP: Deploy

video::zLltQxVb3NI[youtube, start=0]
****

https://12factor.net/[12factor^]


== Development - From Design to Application

=== programming language
==== nodeJS
****
TIP: NodeJS

video::ENrzD9HAZK4[youtube, start=0]
****

https://nodejs.org/en/[nodeJS^]

https://p5js.org/[p5js^]

https://threejs.org/[threejs^]

==== python
https://www.python.org/[python^]

=== Big data and Scientific analysis

https://numpy.org/[numpy^]

https://scipy.org/[scipy^]

https://jupyter.org/[jupyter^]

=== Rest API
****
TIP: Rest api

video::s7wmiS2mSXY[youtube, start=0]
****

=== Sockets
****
TIP: Sockets

video::ZBM28ZPlin8[youtube, start=0]
****

https://socket.io/[socket.io^]



=== Swarm intelligence

==== TensorSwarm: A framework for reinforcement learning of robot swarms.

https://github.com/TensorSwarm/TensorSwarm[TensorSwarm^]


==== ROS - Robot Operating System

https://www.ros.org/[ROS]

==== Reinforcement Learning

https://www.tensorflow.org/agents/tutorials/0_intro_rl[Introduction to RL^]

https://www.tensorflow.org/js/guide/nodejs?hl=es[tensorflow.js^]

https://github.com/karpathy/reinforcejs[common RL algorithms^]

https://developer.ibm.com/tutorials/an-introduction-to-ai-in-nodejs/[An introduction to AI in Node.js^]

https://pytorch.org/[pytorch^]

https://reagent.ai/[pytorch - Reinforcement Learning Platform^]


*PyTorch vs. TensorFlow*

[cols="1,1"]
|===
|PyTorch | TensorFlow

|PyTorch is open source deep learning framework created by developers at Facebook and released in 2017. 
|TensorFlow is open source deep learning framework created by developers at Google and released in 2015. 

|*Top PyTorch Projects*

CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. https://stanfordmlgroup.github.io/projects/chexnet/[url^]

PYRO: Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend. https://pyro.ai/[url^]

Horizon: A platform for applied reinforcement learning (Applied RL) https://horizonrl.com[url^]

|*Top TensorFlow Projects*

Magenta: An open source research project exploring the role of machine learning as a tool in the creative process. https://magenta.tensorflow.org/[url^]

Sonnet: Sonnet is a library built on top of TensorFlow for building complex neural networks. https://sonnet.dev/[url^]

Ludwig: Ludwig is a toolbox to train and test deep learning models without the need to write code. https://uber.github.io/ludwig/[url^]


| *Production Deployment*
PyTorch doesn't provide any framework to deploy models directly on to the web. You'll have to use either Flask or Django as the backend server.
| *Production Deployment*
When it comes to deploying trained models to production, TensorFlow is the clear winner. We can directly deploy models in TensorFlow using TensorFlow serving which is a framework that uses REST Client API.


|*Visualization*
PyTorch developers use Visdom, the features provided by Visdom are very minimalistic and limited.
|*Visualization*
When it comes to visualization of the training process, TensorFlow takes the lead. Visualization helps the developer track the training process and debug in a more convenient way. 
TenforFlow’s visualization library is called TensorBoard. 

|Python
|Tensorflow has APIs for various langages :

*     C (used to develop other APIs in practice, though)
*     C++ (Tensorflow is written in C++ and CUDA)
*     Java
*     GO
*     Rust
*     Haskell (yes, even Haskell)
*     C#
*     Julia
*     R
*     Python
* 
* And, of course, for JavaScript, since 2019.

|*Pytorch Cons:*

*    Third-party needed for visualization.
*    API server needed for production.

|*TensorFlow Cons:*

*    Debugging method.
*    Hard to make quick changes.


|===



== Auth

https://oauth.net/2/[auth2^]

https://oauth.net/code/nodejs/[Nodejs auth library^]

=== Auth server	

https://www.keycloak.org/[auth2 server^]

== Security VPN etc

https://www.wireguard.com/[simply the best VPN out there^]

== POC

=== netflix

****
TIP: Netflix

video::CZ3wIuvmHeM[youtube, start=0]
****

 6:50 monolithic

 8:24 minroservices

 19:00 microservice

 see also 37:00

https://netflix.github.io/[netflix github^]

===	Robots and IoT
https://nodebots.io/[nodebot^]

http://johnny-five.io/[johny-five^]

https://ardupilot.org/[eduplot^]

https://github.com/ArduPilot/ardupilot[ardupilot^]