zeus
3 years ago
11 changed files with 2 additions and 1297 deletions
|
Before Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 10 KiB |
@ -1,352 +0,0 @@ |
|||
= Linux process management ! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
// Metadata: |
|||
:description: Life cycle of a process |
|||
:keywords: ps aux |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Table of contents |
|||
:toclevels: 4 |
|||
:source-highlighter: highlight |
|||
:no-header-footer: |
|||
:nofooter: |
|||
:last-update-label!: |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
.Life cycle of a process |
|||
**** |
|||
|
|||
[.float-group] |
|||
-- |
|||
.fork-exec footnote:[http://www.it.uu.se/education/course/homepage/os/vt18/images/module-2/fork-exec-exit-wait.png[^]] |
|||
image::./fork-exec-exit-wait.png[350,350,role="related thumb right"] |
|||
|
|||
*fork()* is the the system call that the parent process uses to "divide" itself ("fork") into two identical processes. |
|||
|
|||
* After calling fork(), the created child process is an exact copy of the parent except for the return value of the fork() call. |
|||
|
|||
** This includes open files, register state, and all memory allocations, which includes the program's executable code. In some cases the two continue to run the same binary, |
|||
* the child switches to running another binary executable using the *exec()* system call. |
|||
-- |
|||
**** |
|||
|
|||
== Life cycle of a process |
|||
|
|||
|
|||
|
|||
* When a process forks, a complete copy of the executing program is made into the new process. |
|||
** This new process is a child of the parent process, and has a new process identifier (PID). |
|||
|
|||
* The fork() function returns the child's PID to the parent process. |
|||
** The fork() function returns 0 to the child process. |
|||
** This enables the two identical processes to distinguish one another. |
|||
|
|||
|
|||
* The parent process can either continue execution or wait for the child process to complete. |
|||
|
|||
* The child, after discovering that it is the child, replaces itself completely with another program, so that the code and address space of the original program are lost. |
|||
|
|||
* If the parent chooses to wait for the child to die, then the parent will receive the exit code of the program that the child executed. |
|||
|
|||
* To prevent the child becoming a zombie the parent should call wait on its children, either periodically or upon receiving the SIGCHLD signal, which indicates a child process has terminated. |
|||
|
|||
* One can also asynchronously wait on their children to finish, by using a signal handler for SIGCHLD, if they need to ensure everything is cleaned up. |
|||
|
|||
|
|||
|
|||
[NOTE] |
|||
==== |
|||
* When the child process calls exec(), all data in the original program is lost, and it is replaced with a running copy of the new program. |
|||
|
|||
** This is known as overlaying. |
|||
|
|||
* Although all data are replaced, the file descriptors that were open in the parent are closed only if the program has explicitly marked them close-on-exec. |
|||
|
|||
* This allows for the common practice of the parent creating a pipe prior to calling fork() and using it to communicate with the executed program. |
|||
==== |
|||
|
|||
|
|||
=== copy-on-write |
|||
|
|||
All processes are sharing the same set of pages and each one gets its own private copy when it wants to modify a page. |
|||
|
|||
*In such cases, a technique called copy-on-write (COW) is used.* |
|||
|
|||
With this technique, when a fork occurs, the parent process's pages are not copied for the child process. |
|||
|
|||
Instead, the pages are shared between the child and the parent process. |
|||
|
|||
Whenever a process (parent or child) modifies a page, a separate copy of that particular page alone is made for that process (parent or child) which performed the modification. |
|||
|
|||
This process will then use the newly copied page rather than the shared one in all future references. |
|||
The other process (the one which did not modify the shared page) continues to use the original copy of the page (which is now no longer shared). This technique is called copy-on-write since the page is copied when some process writes to it. |
|||
|
|||
|
|||
|
|||
[NOTE] |
|||
==== |
|||
*copy-on-write* is lazy copying, child process copy the page when try to write it. |
|||
|
|||
So basically, after a fork, almost child's memory is shared with parent. |
|||
|
|||
Before any of the processes made, every child process still have some private memory, modified from parent's or new allocating. |
|||
|
|||
That means even without any action the forked child process has some private memory. |
|||
|
|||
We can verify it with |
|||
[source,bash] |
|||
---- |
|||
cat /proc/PID/smaps |
|||
or |
|||
pmap PID |
|||
---- |
|||
==== |
|||
|
|||
[TIP] |
|||
==== |
|||
This operation avoids unnecessary overhead because copying an entire address space is a very slow and inefficient operation which uses a lot of processor time and resources. |
|||
==== |
|||
|
|||
=== Zombie processes |
|||
|
|||
A child process always first becomes a zombie before being removed from the resource table. |
|||
|
|||
When a process ends via exit, all of the memory and resources associated with it are deallocated so they can be used by other processes. |
|||
|
|||
The process's entry in the process table remains. |
|||
|
|||
The parent can read the child's exit status by executing the wait system call, whereupon the zombie is removed. |
|||
|
|||
The wait call may be executed in sequential code, but it is commonly executed in a handler for the SIGCHLD signal, which the parent receives whenever a child has died. |
|||
|
|||
|
|||
[NOTE] |
|||
==== |
|||
In most cases, under normal system operation zombies are immediately waited on by their parent and then reaped by the system – processes that stay zombies for a long time are generally an error and cause a resource leak, but the only resource they occupy is the process table entry – process ID. |
|||
[source,bash] |
|||
---- |
|||
It is not possible to kill such a process with the kill command, because it is already considered dead. |
|||
You can kill the parent process and then the zombie disappears as well. |
|||
|
|||
Zombies can be identified in the output from the Unix ps command by the presence of a "Z" in the "STAT" column |
|||
---- |
|||
==== |
|||
|
|||
|
|||
=== wait |
|||
|
|||
The child process will not be completely removed until the parent process knows of the termination of its child process by the wait() system call. |
|||
|
|||
A process (or task) may wait on another process to complete its execution. |
|||
|
|||
The parent process issue a wait system call, which suspends the execution of the parent process while the child executes. |
|||
|
|||
When the child process terminates, it returns an exit status to the operating system, which is then returned to the waiting parent process. |
|||
|
|||
The parent process then resumes execution. |
|||
|
|||
=== Orphan process |
|||
|
|||
A child process whose parent process terminates before it does becomes an orphan process. |
|||
|
|||
Such situations are typically handled with a special "root" (or "init") process, which is assigned as the new parent of a process when its parent process exits. |
|||
|
|||
This special process detects when an orphan process terminates and then retrieves its exit status, allowing the system to deallocate the terminated child process. |
|||
|
|||
=== Process States |
|||
|
|||
[source,bash] |
|||
---- |
|||
ps aux |
|||
---- |
|||
|
|||
In the *STAT* column, you'll see: |
|||
|
|||
* R: running or runnable, it is just waiting for the CPU to process it |
|||
* S: Interruptible sleep, waiting for an event to complete, such as input from the terminal |
|||
* D: Uninterruptible sleep, processes that cannot be killed or interrupted with a signal, usually to make them go away you have to reboot or fix the issue |
|||
* Z: Zombie, we discussed in a previous lesson that zombies are terminated processes that are waiting to have their statuses collected |
|||
* T: Stopped, a process that has been suspended/stopped |
|||
|
|||
== Thread |
|||
|
|||
A thread is an execution unit that has its own program counter, a stack and a set of registers that reside in a process |
|||
|
|||
|
|||
Multiple threads can exist within one process, executing concurrently and sharing resources such as memory, while different processes do not share these resources. |
|||
|
|||
The threads of a process share its executable code and the values of its dynamically allocated variables and non-thread-local global variables at any given time. |
|||
|
|||
[.float-group] |
|||
-- |
|||
.A process with two threads footnote:[https://commons.wikimedia.org/w/index.php?curid=2233446[^]] |
|||
image::./Multithreaded_process.svg[350,350,role="related thumb right"] |
|||
|
|||
Threads in the same process share: |
|||
|
|||
* Process instructions |
|||
* open files, data |
|||
* signals and signal handlers |
|||
* current working directory |
|||
* User and group id |
|||
-- |
|||
|
|||
NOTE: A thread is also called Light Weight Process (LWP). |
|||
|
|||
=== Implementations |
|||
|
|||
* LinuxThreads |
|||
** The default thread implementation since Linux kernel 2.0 (introduced in 1996) |
|||
* Native POSIX Thread Library (NPTL) |
|||
** NPTL has been part of Red Hat Enterprise Linux since version 3, and in the Linux kernel since version 2.6. It is now a fully integrated part of the GNU C Library. |
|||
* Next Generation POSIX Thread (NGPT) |
|||
** A IBM developed version of POSIX thread library. The NGPT team collaborated closely with the NPTL team and combined the best features of both implementations into NPTL. |
|||
|
|||
|
|||
=== Threads vs. processes pros and cons |
|||
|
|||
|
|||
* processes are typically independent, while threads exist as subsets of a process |
|||
* processes carry considerably more state information than threads, whereas multiple threads within a process share process state as well as memory and other resources |
|||
* processes have separate address spaces, whereas threads share their address space |
|||
* processes interact only through system-provided inter-process communication mechanisms |
|||
* context switching between threads in the same process typically occurs faster than context switching between processes |
|||
|
|||
|
|||
Advantages and disadvantages of threads vs processes include: |
|||
|
|||
* *Lower resource consumption of threads*: using threads, an application can operate using fewer resources than it would need when using multiple processes. |
|||
* *Simplified sharing and communication of threads*: unlike processes, which require a message passing or shared memory mechanism to perform inter-process communication (IPC), threads can communicate through data, code and files they already share. |
|||
|
|||
TIP: *Thread crashes a process*: due to threads sharing the same address space, an illegal operation performed by a thread can crash the entire process; therefore, one misbehaving thread can disrupt the processing of all the other threads in the application. |
|||
|
|||
|
|||
==== When should you prefer fork() over threading and vice-verse? |
|||
|
|||
When you're doing a far more complex task than just instantiating a worker, or you want the implicit security sandboxing of separate processes. |
|||
|
|||
==== If I want to call an external application as a child, then should I use fork() or threads to do it? |
|||
|
|||
If the child will do an identical task to the parent, with identical code, use fork. For smaller subtasks use threads. |
|||
|
|||
==== it is bad thing to call a fork() inside a thread? |
|||
|
|||
it's computationally rather expensive to duplicate a process and a lot of subthreads. |
|||
|
|||
== Process Memory |
|||
|
|||
A process uses its own memory area to perform work. |
|||
|
|||
[.float-group] |
|||
-- |
|||
.Process Memory footnote:[https://gabrieletolomei.files.wordpress.com/2013/10/program_in_memory2.png?w=960[^]] |
|||
image::./program_in_memory2.png[350,350,role="related thumb right"] |
|||
|
|||
* Text Segment. |
|||
** The Text segment (a.k.a the Instruction segment) contains the executable program code and constant data. |
|||
|
|||
* Data Segment |
|||
** Heap |
|||
*** Heap is the segment from which the memory is provided. (e.g. malloc()) |
|||
|
|||
** BSS: |
|||
*** The area where zero-initialized data is stored. All the global variable which are not initialized in the program are stored in the BSS segment. |
|||
|
|||
** Data: |
|||
*** The area where initialized data are stored. |
|||
|
|||
* Stack Segment |
|||
** The stack segment is used by the process for the storage of automatic identifier, register variables, and function call information. |
|||
-- |
|||
|
|||
.what process is attached to a shared memory segment? |
|||
[source,bash] |
|||
---- |
|||
You can use: |
|||
ipcs -mp to get the process ID |
|||
and |
|||
with the command grep [shared memory segment] /proc/*/maps |
|||
---- |
|||
|
|||
TIP: *ipcs* shows information on the inter-process communication facilities for which the calling process has read access. |
|||
By default it shows information about all three resources: shared memory segments, message queues, and semaphore arrays. |
|||
|
|||
== Process priority (nice) |
|||
|
|||
In Linux we can set guidelines for the CPU to follow when it is looking at all the tasks it has to do. |
|||
These guidelines are called niceness or nice value. |
|||
|
|||
The "niceness" scale goes from |
|||
|
|||
* -20 (highest priority value) |
|||
* 19 (lowest priority value) |
|||
* default is 0 |
|||
|
|||
The nice priority is actually used for user programs. |
|||
|
|||
NOTE: Priority is all about managing processor time |
|||
|
|||
[source,bash] |
|||
---- |
|||
nice run a program with modified scheduling priority |
|||
chrt allows to set your scheduling policy as well as priority. |
|||
---- |
|||
== Scheduler |
|||
|
|||
|
|||
[.float-group] |
|||
-- |
|||
.Structure of the Linux kernel footnote:[By ScotXW - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=47075153[^]] |
|||
image::./Simplified_Structure_of_the_Linux_Kernel.svg[350,350,role="related thumb right"] |
|||
|
|||
The scheduler is the Linux kernel part that decides which runnable process will be executed by the CPU next. |
|||
|
|||
It handles CPU resource allocation for executing processes, and aims to maximize overall CPU utilization while also maximizing interactive performance. |
|||
|
|||
|
|||
|
|||
-- |
|||
|
|||
[NOTE] |
|||
==== |
|||
|
|||
The scheduler makes it possible to execute multiple programs at the same time, thus sharing the CPU with users of varying needs. |
|||
|
|||
* Minimizing response time |
|||
* Maximizing overall CPU utilization |
|||
==== |
|||
|
|||
TIP: Since Linux 2.6.23, the default scheduler is CFS, the "Completely Fair Scheduler". The |
|||
CFS scheduler replaced the earlier "O(1)" scheduler. footnote:[https://en.wikipedia.org/wiki/Completely_Fair_Scheduler[^]] footnote:[https://www.kernel.org/doc/html/latest/scheduler/index.html[^]] |
|||
|
|||
|
|||
== Context switching |
|||
|
|||
*Context switch* is the process of storing the state of a process or thread, so that it can be restored and resume execution at a later point. |
|||
This allows multiple processes to share a single central processing unit (CPU), and is an essential feature of a multitasking operating system. |
|||
|
|||
In the Linux kernel, context switching involves |
|||
|
|||
* switching registers |
|||
* stack pointer |
|||
* program counter |
|||
* flushing the translation lookaside buffer (TLB) |
|||
* and loading the page table of the next process to run (unless the old process shares the memory with the new). |
|||
|
|||
== Interrupts |
|||
|
|||
An interrupt is an event that alters the normal execution flow of a program and can be generated by hardware devices or even by the CPU itself. |
|||
|
|||
Interrupts can be grouped into two categories based on the source of the interrupt: |
|||
|
|||
* synchronous, generated by executing an instruction |
|||
* asynchronous, generated by an external event |
|||
** For example a network card generates an interrupts to signal that a packet has arrived. |
|||
|
|||
[source,bash] |
|||
---- |
|||
Information related to hard interrupts at /proc/interrupts |
|||
---- |
|||
|
|||
|
Before Width: | Height: | Size: 19 KiB |
|
Before Width: | Height: | Size: 78 KiB |
|
Before Width: | Height: | Size: 32 KiB |
|
Before Width: | Height: | Size: 37 KiB |
@ -1,45 +0,0 @@ |
|||
[ |
|||
{ |
|||
"chapter": 1, |
|||
"title": "Start a Lab", |
|||
"subtitle": "Absolute basics", |
|||
"type": "video", |
|||
"desc": "Simple Steps to Start Virtual Lab", |
|||
"service":"vimeo", |
|||
"file": "510376917" |
|||
}, |
|||
{ |
|||
"chapter": 2, |
|||
"title": "An Introduction to process management!", |
|||
"subtitle": "Life cycle of a process", |
|||
"type": "asciidoc", |
|||
"desc": "Process creation - fork() or clone() system calls", |
|||
"file": "processes.adoc" |
|||
}, |
|||
{ |
|||
"chapter": 2, |
|||
"title": "An Introduction to Linux !", |
|||
"subtitle": "Absolute basics", |
|||
"type": "asciinema", |
|||
"desc": "Absolute basics, File types, Access rights etc", |
|||
"file": "basics.cast" |
|||
}, |
|||
|
|||
{ |
|||
"chapter": 3, |
|||
"title": "fork-exec", |
|||
"subtitle": "Life cycle of a process", |
|||
"type": "asciidoc", |
|||
"desc": "fork() is the the system call that the parent process uses to 'divide' itself ('fork') into two identical processes.", |
|||
"file": "Life_cycle_of_a_process.adoc" |
|||
}, |
|||
{ |
|||
"chapter": 3, |
|||
"title": "Linux file system layout", |
|||
"subtitle": "Absolute basics", |
|||
"type": "asciinema", |
|||
"desc": "Subdirectories of the root directory", |
|||
"file": "filesystem.cast" |
|||
} |
|||
|
|||
] |
|||
@ -1,112 +0,0 @@ |
|||
= Linux process management ! |
|||
Apostolos rootApostolos@swarmlab.io |
|||
// Metadata: |
|||
:description: Linux process management |
|||
:keywords: ps aux |
|||
:data-uri: |
|||
:toc: right |
|||
:toc-title: Table of contents |
|||
:toclevels: 4 |
|||
:source-highlighter: highlight |
|||
:no-header-footer: |
|||
:nofooter: |
|||
:last-update-label!: |
|||
:icons: font |
|||
:sectnums: |
|||
|
|||
|
|||
|
|||
== process |
|||
|
|||
An instance of a running program is called a process. |
|||
Every time you run a shell command, a program is run and a process is created for it. |
|||
|
|||
|
|||
In any modern operating system there can be more than one instance of a program loaded in memory at the same time. |
|||
|
|||
== Linux process management |
|||
|
|||
Operating systems need some ways to create processes. |
|||
|
|||
* In a very simple system designed for running only a single application (e.g., the controller in a microwave oven), it may be possible to have all the processes that will ever be needed be present when the system comes up. |
|||
* In general-purpose systems, however, some way is needed to create and terminate processes as needed during operation. |
|||
|
|||
[NOTE] |
|||
==== |
|||
|
|||
*There are four principal events that cause a process to be created:* |
|||
|
|||
* System initialization. |
|||
* Execution of process creation system call by a running process. |
|||
* A user request to create a new process. |
|||
* Initiation of a batch job. |
|||
==== |
|||
|
|||
Process creation in UNIX and Linux are done through *fork()* or *clone()* system calls. |
|||
|
|||
There are several steps involved in process creation. |
|||
|
|||
* process scheduling, |
|||
* interrupt handling, |
|||
* signaling, |
|||
* process prioritization, |
|||
* process switching, |
|||
* process state, |
|||
* process memory |
|||
|
|||
=== Life cycle of a process |
|||
|
|||
NOTE: *1->* creation *2->* execution *3->* termination *4->* removal |
|||
|
|||
=== process state |
|||
|
|||
An operating system kernel that allows multitasking needs processes to have certain states. Names for these states are not standardised, but they have similar functionality. |
|||
|
|||
|
|||
[.float-group] |
|||
-- |
|||
[.right] |
|||
image:./Process_states.png[350,350,role="related thumb right"] |
|||
|
|||
* the process is *"created"*. |
|||
* After that the process scheduler assigns it the *"waiting"* state. |
|||
* While the process is *"waiting"*, it waits for the scheduler to do a so-called context switch. |
|||
* The context switch loads the process into the processor and changes the state to *"running"* while the previously *"running"* process is stored in a *"waiting"* state. |
|||
* If a process in the *"running"* state needs to wait for a resource, it is assigned the *"blocked"* state. |
|||
* The process state is changed back to *"waiting"* when the process no longer needs to wait (in a blocked state). |
|||
* Once the process *finishes execution*, or is *terminated* by the operating system, it is no longer needed. |
|||
* The process is *removed instantly* or is moved to the *"terminated"* state. |
|||
* When removed, it just waits to be *removed from main memory*. |
|||
|
|||
|
|||
-- |
|||
|
|||
|
|||
=== Inter-process communication |
|||
|
|||
When processes need to communicate with each other they must share parts of their address spaces or use other forms of inter-process communication (IPC). |
|||
|
|||
* For instance in a shell pipeline, the output of the first process need to pass to the second one. |
|||
* another example is a task that can be decomposed into cooperating but partially independent processes which can run at once |
|||
** (i.e., using concurrency, or true parallelism - the latter model is a particular case of concurrent execution and is feasible whenever enough CPU cores are available for all the processes that are ready to run). |
|||
|
|||
NOTE: It is even possible for two or more processes to be running on different machines that may run different operating system (OS), therefore some mechanisms for communication and synchronization (called communications protocols for distributed computing) are needed (e.g., the *Message Passing Interface, often simply called MPI*). |
|||
|
|||
|
|||
==== communication between processes |
|||
|
|||
Processes can communicate with each other through: |
|||
|
|||
* Shared Memory |
|||
* Message passing |
|||
|
|||
|
|||
[.float-group] |
|||
-- |
|||
[.right] |
|||
image:./1-76.png[350,350,role="related thumb right"] |
|||
[.left] |
|||
image:./2-50.png[350,350,role="related thumb left"] |
|||
-- |
|||
|
|||
footnote:[https://www.geeksforgeeks.org/inter-process-communication-ipc/[^]] |
|||
|
Before Width: | Height: | Size: 77 KiB |
Loading…
Reference in new issue