Exercises:Shared Memory: Difference between revisions

From COMP15212 Wiki
gravatar Yuron [userbureaucratinterface-adminsysopPHRhYmxlIGNsYXNzPSJ0d3BvcHVwIj48dHI+PHRkIGNsYXNzPSJ0d3BvcHVwLWVudHJ5dGl0bGUiPkdyb3Vwczo8L3RkPjx0ZD51c2VyPGJyIC8+YnVyZWF1Y3JhdDxiciAvPmludGVyZmFjZS1hZG1pbjxiciAvPnN5c29wPGJyIC8+PC90ZD48L3RyPjwvdGFibGU+] (talk | contribs)
m (1 revision imported)
pc>Yuron
No edit summary
Line 13: Line 13:
----
----
<blockquote>
<blockquote>
You should have covered [[Exercises:C_Addresses | basic addressing]]
You should have covered [[Exercises:C_Addresses|basic addressing]] and spawning [[Exercises:C_processes|processes]] first.
and spawning [[Exercises:C_processes | processes]] first.
</blockquote>
</blockquote>


=== Part 1 ===
=== Part 1 ===
The processes used here could be separated and executed independently
The processes used here could be separated and executed independently from a shell (or shells) but, for convenience, the <strong>two</strong> processes share a source file and the <code>fork()</code> function is used.
from a shell (or shells) but, for convenience, the <strong>two</strong> processes
share a source file and the <code>fork()</code> function is used.


Compile and execute the example code <code>shared_mem.c</code>.  Understand what
Compile and execute the example code <code>shared_mem.c</code>.  Understand what <code>main</code> is doing here.  You can treat the accompanying calls as ‘magic’ if it makes things easier; they are just wrapping up the C library calls.
<code>main</code> is doing here.  You can treat the accompanying calls as
‘magic’ if it makes things easier; they are just wrapping
up the C library calls.


Two identical values are set up, one in a variable within the process
Two identical values are set up, one in a variable within the process (<code>private</code>) and one referenced in a (potentially) shared memory.  The process then forks.  Each process will now have its own copy of the <code>private</code> variable and its own copy of the <em>pointer</em> to the shared memory.  The addresses seen by each process are in their own virtual memory map: the same virtual address is not necessarily the same physical address and these locations can be modified independently.
(<code>private</code>) and one referenced in a (potentially) shared memory.  The
process then forks.  Each process will now have its own copy of the
<code>private</code> variable and its own copy of the <em>pointer</em> to the shared
memory.  The addresses seen by each process are in their own virtual
memory map: the same virtual address is not necessarily the same
physical address and these locations can be modified independently.


However, the ‘shared’ area in each process has been
However, the ‘shared’ area in each process has been deliberately mapped to the same physical memory (even if the virtual address in each process was different).  Thus if one process changes a value in this space it is changed for all processes sharing this area.
deliberately mapped to the same physical memory (even if the virtual
address in each process was different).  Thus if one process changes a
value in this space it is changed for all processes sharing this area.


In the figure below, the variables <code>private</code> and <code>p_shm</code> will both be
In the figure below, the variables <code>private</code> and <code>p_shm</code> will both be duplicated, in the process’ own data spaces <em>but</em> <code>p_shm</code> is <code>pointing</code> at the shared block, so anything it [[Pointers|references]] will appear to both processes.
duplicated, in the process’ own data spaces <em>but</em> <code>p_shm</code> is
<code>pointing</code> at the shared block, so anything it
[[Pointers | references]] will appear to both processes.


[[Image:shared_memory.png|link=|alt=Shared Memory]]
[[Image:shared_memory.png|link=|alt=Shared Memory]]


One process (the child) changes its variables.  The parent – after a
One process (the child) changes its variables.  The parent – after a little sleep to give the child time to act – prints its variables: <code>private</code> remains stable whereas the shared variable has been changed externally.
little sleep to give the child time to act – prints its variables:
<code>private</code> remains stable whereas the shared variable has been changed
externally.


Note that both parent and child store <code>private</code> at the <strong>same
Note that both parent and child store <code>private</code> at the <strong>same <em>virtual</em> address</strong> but each ends up with a different data value; the processes have their own views virtual memory spaces.
<em>virtual</em> address</strong> but each ends up with a different data value; the
processes have their own views virtual memory spaces.


As always, your own drawing of the memory should help to clarify matters!
As always, your own drawing of the memory should help to clarify matters!


=== Part 2 ===
=== Part 2 ===
In subdirectory <code>message</code>, <code>send.c</code> and <code>receive.c</code> are independent
In subdirectory <code>message</code>, <code>send.c</code> and <code>receive.c</code> are independent programs which will communicate through a shared memory.  There are different ways to do this but probably the easiest is to use two windows, one for each process.  First, compile these programs; then <em>in this order</em>:
programs which will communicate through a shared memory.  There are
different ways to do this but probably the easiest is to use two
windows, one for each process.  First, compile these programs; then
<em>in this order</em>:


*execute <code>receive</code> in one window
*execute <code>receive</code> in one window
Line 70: Line 43:
** in the other window if you have it – it’s somehow more impressive that way.
** in the other window if you have it – it’s somehow more impressive that way.


What you should see is the receiver picking up the message which the
What you should see is the receiver picking up the message which the sender has placed in a shared buffer.  There may be a slight delay because the receiver is polling the memory and having little sleeps in between.
sender has placed in a shared buffer.  There may be a slight delay
because the receiver is polling the memory and having little sleeps in
between.


A point to note: the virtual addresses of the shared memory are
A point to note: the virtual addresses of the shared memory are (probably) completely different, even though the physical memory is
(probably) completely different, even though the physical memory is
clearly the same.
clearly the same.


Now look at the source codes and understand the (crude) protocol used
Now look at the source codes and understand the (crude) protocol used to synchronise the communications.  <strong>Draw</strong> the memory layout. Notice that the two programs use different C syntax to indicate the start of the string: you may like to refer back to [[Extra:Pointer Arithmetic|pointer arithmetic]] if this is not clear. (It is usually baffling at first but your drawing – you did do a drawing? – should help.)
to synchronise the communications.  <strong>Draw</strong> the memory layout.
Notice that the two programs use different C syntax to indicate the
start of the string: you may like to refer back to [[Pointer_Arithmetic | pointer
arithmetic]] if this is not clear.
(It is usually baffling at first but your drawing – you did do a
drawing? – should help.)


=== Something practical ===
=== Something practical ===
Demonstrate your understanding with a small modification to these
Demonstrate your understanding with a small modification to these codes to do something a bit different.  Suggestions might include (one of):
codes to do something a bit different.  Suggestions might include (one
of):


*allow the sending of multiple messages
*allow the sending of multiple messages
Line 99: Line 60:
There are some commented-out lines which might help.
There are some commented-out lines which might help.


Note that, as supplied, <code>receive</code> (only) destroys the buffer when it
Note that, as supplied, <code>receive</code> (only) destroys the buffer when it completes and it ought to shut down tidily to do this or the shared memory will get left behind.  You may want a means of signalling it to stop.
completes and it ought to shut down tidily to do this or the shared
memory will get left behind.  You may want a means of signalling it to
stop.


Also note that, as supplied, the buffer size is only 100 bytes; you
Also note that, as supplied, the buffer size is only 100 bytes; you can alter this as long as you do it consistently.  You can even make extra shared memories using different ‘key’ values.
can alter this as long as you do it consistently.  You can even make
extra shared memories using different ‘key’ values.
<blockquote>
<blockquote>
You may get into trouble crashing and leaving a disconnected area of
You may get into trouble crashing and leaving a disconnected area of shared memory in the computer.  Remember that by necessity, a shared memory cannot ‘belong’ to a single process so it cannot be cleared up automatically when one process finishes.  Hopefully
shared memory in the computer.  Remember that by necessity, a shared
the software supplied is fairly robust at sorting this out.  If there are problems, try the utility <code>clean</code> included here which should find and destroy a detached, shared memory with the supplied key and size.
memory cannot ‘belong’ to a single process so it cannot
be cleared up automatically when one process finishes.  Hopefully
the software supplied is fairly robust at sorting this out.  If
there are problems, try the utility <code>clean</code> included here which
should find and destroy a detached, shared memory with the supplied
key and size.
</blockquote>
</blockquote>
A few people have asked why the polling loops, e.g.:
A few people have asked why the polling loops, e.g.:
Line 158: Line 108:


----
----
=== Submission ===
* Your development of the source files, combined into a single text file [<code>ex6.txt</code>].  Please include a <em>short</em> (1 or 2 sentence) explanation of what you were doing at the front of this file.
----
{{PageGraph}}
{{PageGraph}}
{{Category|Exercises}}
{{Category|Exercises}}

Revision as of 10:14, 8 August 2019

On path: Exercises 0: Exercises • 1: Pointer Exercise • 2: Arguments Exercise • 3: Malloc Exercise • 4: Structs Exercise • 5: Processes Exercise • 6: Shared memory Exercise • 7: Pipes Exercise • 8: Exceptions Exercise • 9: Synchronisation Exercise • 10: Files Exercise • 11: Threads Exercise • 12: Unix proc Exercise
Depends on Memory ManagementShared Memory

Download exercise files

The purposes of this exercise are:

  • to reinforce the understand of memory mapping
  • to illustrate practically the principle of processes sharing memory
  • to explore the principle of cyclic buffers

You should have covered basic addressing and spawning processes first.

Part 1

The processes used here could be separated and executed independently from a shell (or shells) but, for convenience, the two processes share a source file and the fork() function is used.

Compile and execute the example code shared_mem.c. Understand what main is doing here. You can treat the accompanying calls as ‘magic’ if it makes things easier; they are just wrapping up the C library calls.

Two identical values are set up, one in a variable within the process (private) and one referenced in a (potentially) shared memory. The process then forks. Each process will now have its own copy of the private variable and its own copy of the pointer to the shared memory. The addresses seen by each process are in their own virtual memory map: the same virtual address is not necessarily the same physical address and these locations can be modified independently.

However, the ‘shared’ area in each process has been deliberately mapped to the same physical memory (even if the virtual address in each process was different). Thus if one process changes a value in this space it is changed for all processes sharing this area.

In the figure below, the variables private and p_shm will both be duplicated, in the process’ own data spaces but p_shm is pointing at the shared block, so anything it references will appear to both processes.

Shared Memory

One process (the child) changes its variables. The parent – after a little sleep to give the child time to act – prints its variables: private remains stable whereas the shared variable has been changed externally.

Note that both parent and child store private at the same virtual address but each ends up with a different data value; the processes have their own views virtual memory spaces.

As always, your own drawing of the memory should help to clarify matters!

Part 2

In subdirectory message, send.c and receive.c are independent programs which will communicate through a shared memory. There are different ways to do this but probably the easiest is to use two windows, one for each process. First, compile these programs; then in this order:

  • execute receive in one window
    • if you are using a single window, detach this from the parent shell with receive &.
  • execute send <message> where the message is a single word of your choice.
    • in the other window if you have it – it’s somehow more impressive that way.

What you should see is the receiver picking up the message which the sender has placed in a shared buffer. There may be a slight delay because the receiver is polling the memory and having little sleeps in between.

A point to note: the virtual addresses of the shared memory are (probably) completely different, even though the physical memory is clearly the same.

Now look at the source codes and understand the (crude) protocol used to synchronise the communications. Draw the memory layout. Notice that the two programs use different C syntax to indicate the start of the string: you may like to refer back to pointer arithmetic if this is not clear. (It is usually baffling at first but your drawing – you did do a drawing? – should help.)

Something practical

Demonstrate your understanding with a small modification to these codes to do something a bit different. Suggestions might include (one of):

  • allow the sending of multiple messages
    • either by running send repeatedly
    • or by looping in send
  • return a reply message from receive to send.

There are some commented-out lines which might help.

Note that, as supplied, receive (only) destroys the buffer when it completes and it ought to shut down tidily to do this or the shared memory will get left behind. You may want a means of signalling it to stop.

Also note that, as supplied, the buffer size is only 100 bytes; you can alter this as long as you do it consistently. You can even make extra shared memories using different ‘key’ values.

You may get into trouble crashing and leaving a disconnected area of shared memory in the computer. Remember that by necessity, a shared memory cannot ‘belong’ to a single process so it cannot be cleared up automatically when one process finishes. Hopefully the software supplied is fairly robust at sorting this out. If there are problems, try the utility clean included here which should find and destroy a detached, shared memory with the supplied key and size.

A few people have asked why the polling loops, e.g.:

while (p_shm[0] == 0) sleep(1);

contain the sleep(1) statement.


Why?

Firstly, this will deschedule that process for 1 s, which saves wasting processor time.

However removing this also causes the software to fail completely – at least on some systems. Here is the explanation and a fix.

while (p_shm[0] == 0) ;

is a visibly empty loop and a compiler is likely to spot this. It is then likely to optimise the code to reduce the number of (slow) memory loads. This means that it will fetch the value from the shared memory once and then keep looking at that value; it won’t see when it changes.

One way to address this might be to turn the compiler optimisation off. Try altering the makefile to:

CFLAGS=-g -O0

A better way is to force the code to read just this variable when it needs to. This is accommodated in C by declaring a variable as volatile. This is typically used to mark I/O device registers etc. The shared memory locations – which are modified by another process – are, effectively, input devices.

volatile unsigned char* p_shm;

seems to work, but on gcc (at least) produces some compiler ‘warning’ messages. One way to suppress these is to declare a second, ‘volatile’pointer, with a copy of the value simply for use in the polling loop. Thus:

volatile unsigned char* p_copy;

...

p_copy = &p_shm[0];         // Same as "p_copy = p_shm;" but made explicit for emphasis.

...

while (*p_copy == 0) ;      // Poll for message