Security

Depends on	What is an OS

Computer security is a pervasive topic. Most computer systems are open to various forms of insecurity, ranging from user errors to malicious attacks.

With a single-user system – in a locked room – security is not the greatest worry. As more features are shared – processors, memory, files – and machines become more interconnected there is an increasing number of interactions which can allow security breaches.

It is important to offer protection at various levels of a system. In an operating system the most important areas are separating users and processes from each other in the memory and the file-store. Users are also isolated from changing the system itself by various privilege levels.

Many of the older techniques used in operating system security are primarily present to prevent inadvertent problems, such as a crashing program destroying other processes’ memory images or the accidental deletion of files. However there are an increasing number of techniques being introduced to make life difficult for hackers.

There is not the space here for a wide review of computer vulnerabilities. The subjects below are some issues with particular relevance to O.S. security.

---

Access security

The computer account

A subject in its own right, we can only look at some of the principles here. One of these which occurs early on is the log-in password.

It used to be the case in Unix systems that the passwd file was readable by anyone on the system. This contains the username, password, user ID, home directory … The reason it is (reasonably) safe is that the password is stored in encrypted form.

Strictly, this “encryption” is a hash function: it is not intended to be reversed.

The hash function (form of encryption) is known to anyone who cares, but it is a one-way function – very hard to reverse. This is okay because the typed password either matches or not and thus the encryption output will match … or not. It is almost impossible to guess an input which will produce a desired output.

The first weakness here is that computer power gets ever cheaper and an attacker can pre-calculate the output of a set of likely passwords (start with an on-line dictionary) and then – assuming (s)he has got the password file – search to see if anyone is using them. That user account can then be used for further access.

Limiting access to the password file is one more security measure; another is salting the file: including an extra substring which is concatenated with the password before encryption (which means the attacker has to calculate and try many many more possible candidate passwords. It doesn’t guarantee (s)he can’t get a hit but it makes it a lot more work.

No cracker is going to type all this by hand. To limit automated attacks it is possible to disallow attempts to log into an account (for a time) after a number of failed attempts. This slows down an attacker and may make it infeasible to break in this way before (s)he dies of old age.

If you are interested in Security Engineering you could do worse than look in this book which is available free, on line. (It isn’t focussed on operating systems though.)

Firewalls

In these days of networked computers it is not just a physically present hacker who needs repelling. A firewall is a boundary defence, typically separating a computer or Local Area Network (LAN) from another network, such as the Internet.

The firewall – which may be software, hardware, or some combination – examines network packets and only allows approved ones through. This can apply to both incoming and outgoing traffic. Thus, for example, interactions on the LAN may be possible which cannot be done from ‘outside’.

Files

Files are a security hazard because they are potentially shared amongst users. This is desirable in many cases, such as binary files: imagine if every user had a private copy of every utility (filling up the disk) and then there was a software update (find & alter every copy). However it would be bad security if everyone could write to shared utility as this is an easy way to get someone to unwittingly run “malware”.

Files are typically protected with some access control although these were often originally designed to resist casual or accidental problems, not deliberate attack. Stricter and more flexible systems, such as access control lists, are gradually becoming more common.

File permissions are enforced by operating system software as part of the filing system. This can check which user owns a given process and whether a particular file is available for that user. This may also take into account the actions which a user can perform. A typical, simple file setting might be for the creator of a file to retain both read and write access but only allow other users of that filing system read access.

Unix file permissions have three permitted actions {read, write, execute} which are applied to three classes of user {owner, group, anyone}. Group access operates by giving each file a group ID (GID) value; each user belongs to a set of groups configured by the system administrator. The file owner can set the GID. Group access allows the sharing of particular sets of files – e.g. amongst a team working on a collaborative project – without the need for everyone to see what is going on.

This simplistic (single) group permission is a bit simplistic; there is an increasing trend towards more flexible access control lists.

Unix’ “everything is a file” approach means that these permissions are also applicable to other items which appear in the file system tree.

---

Process and memory security

This is discussed in more detail in articles such as “processor privilege” and the MMU. For performance reasons they are implemented in hardware so that an application process can run on the ‘bare’ hardware yet be trapped from doing anything dangerous. Thus, attempts to access parts of the machine which the user is not permitted to do result in an exception trapping back into operating system software.

Examples of such actions may be:

• attempts to read or write to addresses reserved for the operating system, including areas containing items such as:
  • page tables
  • I/O devices
  • O.S. data records
  • O.S. code
• attempts to change the privilege level itself
• attempts to enable/disable interrupts

A particular processor may have several privilege layers. For general operating system support it is usual to have user and supervisor privileges. More sophisticated processors may support virtualisation of the operating systems itself; to be completely transparent this needs a hypervisor privilege layer too, as the operating system(s) is (are) now disconnected from the hardware by further software but they are not aware of this.

---

setuid bit

Sometimes it is important for a user to be able to use a facility – such as an I/O device – which that user is not permitted access to. A solution in Unix is to execute a file which has the setuid bit set. This executes the file with the User ID (UID) of the file owner rather than the user who started it; that owner could be the ‘root’ (administrator). All the access is still done by trusted code, even though a (possibly untrusted!) user invoked this.

See the setuid article for more details.

---

Buffer overflow attack

In brief, a process will have permission to do a certain set of operations. If – either through oversight or laziness or, possibly, deliberately – the original author has left a vulnerability where input data can cause something unpredicted to happen then an attacker may exploit this.

One way to get at variables might be to use a pointer which has a value which it shouldn’t have. This may be possible by loading a string (strings have indefinite length) into a space which is too small for it. The pointer can then go on to allow the overwriting of other locations.

If the attacker knows what these values are – this is perfectly possible in the case of a particular program as (s)he can analyse a local copy – then variables which shouldn’t be accessible can be written to particular values.

This is particularly sensitive if the buffer is on the stack (i.e. declared as a local variable) because the stack also contains procedure return addresses which could be altered to take control of execution. Code could be downloaded into the same buffer at the same time. Gotcha!

See the buffer overflow article for more details.

---

Denial of Service

A simple way to ‘break’ a computer system is to overload it with work. This is the basis of of a “denial of service” attack – pester a legitimately offered service to overload the server. This is usually a network problem.

Across a network, such attacks can be aggravated by using many (often previously compromised) computers to send the incoming requests: a “Distributed Denial of Service” (DDoS) attack.

There are various defences, typically involving identifying sources and rationing the services granted to any particular source. This may be done in combination with a firewall. This is harder if the attacks come from multiple places, i.e. DDoS.

A similar technique may be used to slow down potential password (etc.) cracking attacks: limit the number of tries available before denying the ‘user’ for a period. This limits ‘brute force’ attacks by machines whilst (mostly)

allowing legitimate attempts to log on.

---

Unintended consequences

A high-profile problem emerged into the international media in January 2018. Here security problems emerged as it was revealed that optimisations in one area potentially allowed information to leak elsewhere.

Usually ‘obvious’ vulnerabilities are spotted and allowed for; it is the complex interactions in engineered systems which sometimes escape – at least until there is an exploitation or an accident. This is why holistic approach to a subject is important. (Sermon over.)

---

Remember it only takes one gap in a fence to render it useless.

---

Also refer to:	Operating System Concepts, 10th Edition: Chapter 1.6, 16, pages 33-34, 621-665

Context Dependencies Dependants

Max. number:

Max. distance:

Next

Articles on Security

Buffer Overflow • Containers • Context • Disk Partition • File Attributes • File Permissions • Journalling File System • Mutual exclusion • Processor Privilege • SETUID • Security