Documentation/tech-list/modules/ROOT/pages/index.adoc

= List!
  
image::ROOT:swarmlab.png[swarmlab,150,float=right]


== Why Docker

https://www.cloudbees.com/blog/why-dockero[Why Docker^]

== Intro docker

https://www.compose-spec.io/[docker-compose^]

https://github.com/veggiemonk/awesome-docker[github docker-compose^]

https://twitter.com/awesome_docker[github awesome docker]

=== Learn to build and deploy your distributed applications easily to the cloud with Docker 

https://docker-curriculum.com/[HowTo]

== Next Step DockerSwarm

****
TIP: DockerSwarm

video::Tm0Q5zr3FL4[youtube, start=0]
****

https://www.simplilearn.com/tutorials/docker-tutorial/docker-swarm[docker-swarm tutorial^]

=== Getting started with swarm - official from docker

https://docs.docker.com/engine/swarm/swarm-tutorial/https://k21academy.com/docker-kubernetes/docker-swarm/[docker-swarm^]

=== Replicability, use the same files as when developing locally.

https://dockerswarm.rocks/[dockerswarm.rocks^]

=== Open Source Cloud Guide από την IBM

https://open-cloud-guide.dev/[open-cloud huide^]

=== Alternatives 

https://www.nomadproject.io/[monad^]

https://dev.to/stevenmcgown/kubernetes-for-dummies-5hmh[kubernetes^]

=== Raft consensus in swarm mode

https://docs.docker.com/engine/swarm/raft/[raft^]

https://raft.github.io/[raft github^]

http://thesecretlivesofdata.com/raft/[raft demo^]

== Next Step Storage

****
TIP: Storage

video::W2Z7fbCLSTw[youtube, start=0]
****

=== Generic

https://docs.mongodb.com/manual/replication/[Mongo replica^]

https://docs.mongodb.com/manual/core/capped-collections/[mongo capped^]

=== On memory - Key-value

https://redis.io/topics/cluster-tutorial[Redis cluster^]

https://redis.io/[redis server^]

https://www.instaclustr.com/blog/redis-vs-memcached/[redis vs memcached^]

https://etcd.io/[etcd^]

****
TIP: etcd

video::OmphHSaO1sE[youtube, start=0]
****

****
TIP: redis

video::G1rOthIU-uo[youtube, start=0]
****

*etcd vs. Redis*

Like etcd, Redis is an open source tool, but their basic functionalities are different.

[cols="1,1"]
|===
|Redis|etcd

| Redis is an in-memory data store and can function as a database, cache, or message broker. Redis supports a wider variety of data types and structures than etcd and has much faster read/write performance.
| etcd has superior fault tolerance, stronger failover and continuous data availability capabilities. 

|Redis is better suited for serving as a distributed memory caching system than for storing and distributed system configuration information.
|etcd persists all stored data to disk. 


|===





=== Sharing

==== NFS	

https://cloud.netapp.com/blog/azure-anf-blg-linux-nfs-server-how-to-set-up-server-and-client[NFS^]

https://www.gluster.org/[GlusterFS^]	   


== Deploy

https://www.compose-spec.io/[docker-compose^]

****
TIP: Deploy

video::zLltQxVb3NI[youtube, start=0]
****

https://12factor.net/[12factor^]


== Development - From Design to Application

=== programming language
==== nodeJS
****
TIP: NodeJS

video::ENrzD9HAZK4[youtube, start=0]
****

https://nodejs.org/en/[nodeJS^]

https://p5js.org/[p5js^]

https://threejs.org/[threejs^]

==== python
https://www.python.org/[python^]

=== Big data and Scientific analysis

https://numpy.org/[numpy^]

https://scipy.org/[scipy^]

https://jupyter.org/[jupyter^]

=== Rest API
****
TIP: Rest api

video::s7wmiS2mSXY[youtube, start=0]
****

=== Sockets
****
TIP: Sockets

video::ZBM28ZPlin8[youtube, start=0]
****

https://socket.io/[socket.io^]



=== Swarm intelligence

==== TensorSwarm: A framework for reinforcement learning of robot swarms.

https://github.com/TensorSwarm/TensorSwarm[TensorSwarm^]


==== ROS - Robot Operating System

https://www.ros.org/[ROS]

==== Reinforcement Learning

https://www.tensorflow.org/agents/tutorials/0_intro_rl[Introduction to RL^]

https://www.tensorflow.org/js/guide/nodejs?hl=es[tensorflow.js^]

https://github.com/karpathy/reinforcejs[common RL algorithms^]

https://developer.ibm.com/tutorials/an-introduction-to-ai-in-nodejs/[An introduction to AI in Node.js^]

https://pytorch.org/[pytorch^]

https://reagent.ai/[pytorch - Reinforcement Learning Platform^]


*PyTorch vs. TensorFlow*

[cols="1,1"]
|===
|PyTorch | TensorFlow

|PyTorch is open source deep learning framework created by developers at Facebook and released in 2017. 
|TensorFlow is open source deep learning framework created by developers at Google and released in 2015. 

|*Top PyTorch Projects*

* CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. https://stanfordmlgroup.github.io/projects/chexnet/[url^]

* PYRO: Pyro is a universal probabilistic programming language (PPL) written in Python and supported by PyTorch on the backend. https://pyro.ai/[url^]

* Horizon: A platform for applied reinforcement learning (Applied RL) https://horizonrl.com[url^]

|*Top TensorFlow Projects*

* Magenta: An open source research project exploring the role of machine learning as a tool in the creative process. https://magenta.tensorflow.org/[url^]

* Sonnet: Sonnet is a library built on top of TensorFlow for building complex neural networks. https://sonnet.dev/[url^]

* Ludwig: Ludwig is a toolbox to train and test deep learning models without the need to write code. https://uber.github.io/ludwig/[url^]


| *Production Deployment*
PyTorch doesn't provide any framework to deploy models directly on to the web. You'll have to use either Flask or Django as the backend server.
| *Production Deployment*
When it comes to deploying trained models to production, TensorFlow is the clear winner. We can directly deploy models in TensorFlow using TensorFlow serving which is a framework that uses REST Client API.


|*Visualization*
PyTorch developers use Visdom, the features provided by Visdom are very minimalistic and limited.
|*Visualization*
When it comes to visualization of the training process, TensorFlow takes the lead. Visualization helps the developer track the training process and debug in a more convenient way. 
TenforFlow’s visualization library is called TensorBoard. 

|Python
|Tensorflow has APIs for various langages :

*     C (used to develop other APIs in practice, though)

*     C++ (Tensorflow is written in C++ and CUDA)

*     Java

*     GO

*     Rust

*     Haskell (yes, even Haskell)

*     C#

*     Julia

*     R

*     Python

* And, of course, for JavaScript, since 2019.

|*Pytorch Cons:*

*    Third-party needed for visualization.

*    API server needed for production.

|*TensorFlow Cons:*

*    Debugging method.

*    Hard to make quick changes.


|===


=== Reasons to learn machine learning with JavaScript

====
*Private machine learning*

Most machine learning applications rely on client-server architectures. Users must send their data where the machine learning models are running.

Machine learning models on your device without sending any data to the cloud. 

And you don’t need to install any additional software. 

Powerful JavaScript machine learning libraries:

* https://ml5js.org/[ml5js^]

* http://caza.la/synaptic/#/[synaptic^]

* https://brain.js.org/#/[brain^]


*Easy integration of machine learning in web and mobile applications*

Python support in mobile operating systems is still in the preliminary stages. 

Javascript has a rich set of cross-platform mobile app development tools such as Cordova and Ionic


TIP: Machine learning with Node.js is fairly new, but it is fast evolving because there is growing interest in adding machine learning capabilities to web and mobile applications. 


====


== Auth

https://oauth.net/2/[auth2^]

https://oauth.net/code/nodejs/[Nodejs auth library^]

=== Auth server	

https://www.keycloak.org/[auth2 server^]

== Security VPN etc

https://www.wireguard.com/[simply the best VPN out there^]

== POC

=== netflix

****
TIP: Netflix

video::CZ3wIuvmHeM[youtube, start=0]
****

 6:50 monolithic

 8:24 minroservices

 19:00 microservice

 see also 37:00

https://netflix.github.io/[netflix github^]

===	Robots and IoT
https://nodebots.io/[nodebot^]

http://johnny-five.io/[johny-five^]

https://ardupilot.org/[eduplot^]

https://github.com/ArduPilot/ardupilot[ardupilot^]