You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
255 lines
5.5 KiB
255 lines
5.5 KiB
= Guide to open source development in the cloud!
|
|
|
|
|
|
An https://developer.ibm.com/blogs/oreilly-open-source-skill-survey-blog/[O’Reilly survey^], commissioned by IBM in Fall 2020, pointed out that, when choosing cloud providers, 70% of respondents prefer one based on open source. The survey went on to say:
|
|
|
|
TIP: Developers who want to develop cloud applications should work on honing their open source skills, which underly every major cloud platform, instead of only focusing on building skills for a proprietary cloud.
|
|
|
|
The most desirable open source skills are those related to *Linux (containers)*, *artificial intelligence* and *machine learning*, and *data storage.*
|
|
|
|
This goals of this guide(https://open-cloud-guide.dev/[Text^] by IBM / Apache Licensed, 2021) are to provide:
|
|
|
|
* Comprehensive learning for hybrid cloud and multicloud developers.
|
|
* A cloud-neutral view of how various clouds are using open source in their offerings.
|
|
|
|
====
|
|
****
|
|
TIP: Why Open Source!
|
|
|
|
video::a8fHgx9mE5U[youtube, start=0]
|
|
****
|
|
Open source refers to an *openly distributed* program code (including codes that are *free of charge*) that can be *utilized* and *modified* by the end user *without restriction.*
|
|
====
|
|
|
|
|
|
== Docker
|
|
|
|
* https://www.cloudbees.com/blog/why-docker[Docker^]
|
|
|
|
== Intro docker
|
|
|
|
* https://www.compose-spec.io/[docker-compose^]
|
|
|
|
* https://github.com/veggiemonk/awesome-docker[github docker-compose^]
|
|
|
|
* https://twitter.com/awesome_docker[github awesome docker]
|
|
|
|
=== Learn to build and deploy your distributed applications easily to the cloud with Docker
|
|
|
|
* https://docker-curriculum.com/[HowTo]
|
|
|
|
== Alternatives to Docker
|
|
|
|
TIP: https://apptainer.org/docs/admin/main/[Apptainer/Singularity] - Best Container system for HPC
|
|
|
|
====
|
|
Singularity can be used with Docker images. This feature was included because developers use and really like using Docker and scientists have already put much resources into creating Docker images.
|
|
|
|
Features:
|
|
|
|
* Use a Docker image instantly as a Singularity image
|
|
* Pull a Docker image
|
|
* Build images with bases from assembled Docker layers
|
|
|
|
====
|
|
|
|
|
|
|
|
|
|
== Next Step DockerSwarm
|
|
|
|
****
|
|
TIP: DockerSwarm
|
|
|
|
video::Tm0Q5zr3FL4[youtube, start=0]
|
|
****
|
|
|
|
* https://www.simplilearn.com/tutorials/docker-tutorial/docker-swarm[docker-swarm tutorial^]
|
|
|
|
=== Getting started with swarm - official from docker
|
|
|
|
* https://docs.docker.com/engine/swarm/swarm-tutorial/https://k21academy.com/docker-kubernetes/docker-swarm/[docker-swarm^]
|
|
|
|
=== Replicability, use the same files as when developing locally.
|
|
|
|
* https://dockerswarm.rocks/[dockerswarm.rocks^]
|
|
|
|
|
|
=== Alternatives
|
|
|
|
* https://www.nomadproject.io/[monad^]
|
|
|
|
* https://dev.to/stevenmcgown/kubernetes-for-dummies-5hmh[kubernetes^]
|
|
|
|
=== Raft consensus in swarm mode
|
|
|
|
* https://docs.docker.com/engine/swarm/raft/[raft^]
|
|
|
|
* https://raft.github.io/[raft github^]
|
|
|
|
* http://thesecretlivesofdata.com/raft/[raft demo^]
|
|
|
|
== Next Step Storage
|
|
|
|
****
|
|
TIP: Storage
|
|
|
|
video::W2Z7fbCLSTw[youtube, start=0]
|
|
****
|
|
|
|
=== Generic
|
|
|
|
* https://docs.mongodb.com/manual/replication/[Mongo replica^]
|
|
|
|
* https://docs.mongodb.com/manual/core/capped-collections/[mongo capped^]
|
|
|
|
=== On memory - Key-value
|
|
|
|
* https://redis.io/topics/cluster-tutorial[Redis cluster^]
|
|
|
|
* https://redis.io/[redis server^]
|
|
|
|
* https://www.instaclustr.com/blog/redis-vs-memcached/[redis vs memcached^]
|
|
|
|
* https://etcd.io/[etcd^]
|
|
|
|
****
|
|
TIP: etcd
|
|
|
|
video::OmphHSaO1sE[youtube, start=0]
|
|
****
|
|
|
|
****
|
|
TIP: redis
|
|
|
|
video::G1rOthIU-uo[youtube, start=0]
|
|
****
|
|
|
|
*etcd vs. Redis*
|
|
|
|
Like etcd, Redis is an open source tool, but their basic functionalities are different.
|
|
|
|
[cols="1,1"]
|
|
|===
|
|
|Redis|etcd
|
|
|
|
| Redis is an in-memory data store and can function as a database, cache, or message broker. Redis supports a wider variety of data types and structures than etcd and has much faster read/write performance.
|
|
| etcd has superior fault tolerance, stronger failover and continuous data availability capabilities.
|
|
|
|
|Redis is better suited for serving as a distributed memory caching system than for storing and distributed system configuration information.
|
|
|etcd persists all stored data to disk.
|
|
|
|
|
|
|===
|
|
|
|
|
|
=== Sharing
|
|
|
|
==== NFS
|
|
|
|
* https://cloud.netapp.com/blog/azure-anf-blg-linux-nfs-server-how-to-set-up-server-and-client[NFS^]
|
|
|
|
* https://www.gluster.org/[GlusterFS^]
|
|
|
|
* https://github.com/minio/minio[Minio^]
|
|
|
|
== Deploy
|
|
|
|
* https://www.compose-spec.io/[docker-compose^]
|
|
|
|
****
|
|
TIP: Deploy
|
|
|
|
video::zLltQxVb3NI[youtube, start=0]
|
|
****
|
|
|
|
https://12factor.net/[12factor^]
|
|
|
|
== Development - From Design to Application
|
|
|
|
=== programming language
|
|
|
|
==== nodeJS
|
|
****
|
|
TIP: NodeJS
|
|
|
|
video::ENrzD9HAZK4[youtube, start=0]
|
|
****
|
|
|
|
* https://nodejs.org/en/[nodeJS^]
|
|
|
|
==== python
|
|
|
|
* https://www.python.org/[python^]
|
|
|
|
=== Big data and Scientific analysis
|
|
|
|
* https://numpy.org/[numpy^]
|
|
|
|
* https://scipy.org/[scipy^]
|
|
|
|
* https://jupyter.org/[jupyter^]
|
|
|
|
=== Html5 graphical librarys
|
|
|
|
* https://p5js.org/[p5js^]
|
|
|
|
* https://threejs.org/[threejs^]
|
|
|
|
=== Rest API
|
|
****
|
|
TIP: Rest api
|
|
|
|
video::s7wmiS2mSXY[youtube, start=0]
|
|
****
|
|
|
|
=== Sockets
|
|
****
|
|
TIP: Sockets
|
|
|
|
video::ZBM28ZPlin8[youtube, start=0]
|
|
****
|
|
|
|
* https://socket.io/[socket.io^]
|
|
|
|
|
|
== Auth
|
|
|
|
* https://oauth.net/2/[auth2^]
|
|
|
|
* https://oauth.net/code/nodejs/[Nodejs auth library^]
|
|
|
|
=== Auth server
|
|
|
|
* https://www.keycloak.org/[auth2 server^]
|
|
|
|
== Security VPN etc
|
|
|
|
* https://www.wireguard.com/[simply the best VPN out there^]
|
|
|
|
== POC
|
|
|
|
=== netflix
|
|
|
|
****
|
|
TIP: Netflix
|
|
|
|
video::CZ3wIuvmHeM[youtube, start=0]
|
|
****
|
|
|
|
6:50 monolithic
|
|
|
|
8:24 minroservices
|
|
|
|
19:00 microservice
|
|
|
|
see also 37:00
|
|
|
|
https://netflix.github.io/[netflix github^]
|
|
|
|
|
|
== Open Source Cloud Guide from IBM
|
|
|
|
* https://open-cloud-guide.dev/[open-cloud guide^]
|
|
|
|
|
|
|