Almost every form of digital information exchange can introduce communication errors. Sometimes, these errors can be ignored (for example, an erroneous pixel in a high-resolution video is merely unnoticeable). However, most of the time, they cannot be ignored, and we want to ensure that the receiver gets an absolutely correct stream of information.

In order to overcome the inherent inaccuracy of information transmission, a few methods for error detection and correction have been developed. Generally speaking, these methods introduce some redundancy to the actual message, which in turn can be used to detect errors and in some cases to recover the original data.

One of the most common methods is the use of the CRC (cyclic redundancy check) function, a family of codes commonly used to ensure data integrity in digital data streams by detecting errors due to noise or dropouts of bits in the message stream. The CRC calculates a series of bits (also commonly referred to as a CRC) that is appended to the data stream and transmitted along with the message. The receiver performs a CRC function on the message and compares the result with the received CRC code to verify the integrity of the message. CRC is commonly used in a number of applications such as digital communications and computer data storage systems.

The CRC is performed through binary polynomial division between the transmitted message and a polynomial divisor and is usually implemented using a linear feedback shift register (LFSR). An LFSR is a shift register where its next state is a linear combination of its previous state and input bits. In our context, the linear operators are Logical XOR and Logical AND. Since the operation of an LFSR circuit is deterministic, and the CRC is shorter than the message, typically few messages are mapped to each CRC value. A well-chosen polynomial ensures an evenly distributed mapping of messages to CRC values (for example, if all messages were mapped to the same CRC, detection of an error in a message bit would be impossible).

The trick in an embedded systems design is to implement this technique in the most efficient way possible and in the smallest possible footprint. In this article, we discuss aspects of the theory and implementation of LFSRs and CRCs, illustrated using a family of instructions on Analog Devices’ Blackfin processor specifically defined for addressing the task of efficient LFSR implementation. We also provide a method for converting an implementation from an Internal LFSR form to an External LFSR form.

Links: http://embedded.com/design/206901030

Though we all have a gut-sense that working too many hours is counterproductive, a very short paper by John Nevison called “Overtime Hours: The Rule of Fifty” at Project Solutions (where you need to register) cites data from four studies conducted over a half century to show how productivity either drops, or at best maxes out, as overtime increases.

The studies’ results vary quite a bit. One shows that at 50 hours/week workers do about 37 hours work, dropping to just over 30 once the workweek increases to 55. The “best,” if you can call it that, results were from a 1997 survey showed wielding the whip can have ever-increasing productive rewards, edging up to about 52 productive hours for a 70 hour week. But they all show a nearly-impenetrable barrier of around 50 useful hours or less, regardless of overtime.

Unsurprisingly the data shows a sharp drop in results for those working excessive OT for weeks on end, averaging around a 20% drop after 12 weeks of servitude. That means, as the author concludes, the Rule of Fifty is a best case estimate.

The 2005 Circadian Technologies Shiftware Practices survey showed that productivity can decrease by as much as 25% for a 60 hour workweek, which jibes pretty well with Nevison’s data. Circadian’s results also demonstrate that turnover is nearly three times higher among workers putting in a lot of OT, and absenteeism is twice the national average. I’m not sure what that result means, since it’s awfully hard for an absent worker to be putting in overtime.

Fred Brooks claims that the average software engineer devotes about 55% of his week to project work. The rest goes to overhead activities, responding to HR, meetings about the health insurance plan, and supporting other activities.

The German Embassy’s Washington web site claims on its web site that the nominal workweek in Germany is 37.5 hours because “The original reason for introducing this system was to combat rush-hour traffic congestion, but among the more direct gains are an improvement in employee morale, greater productivity, significant decreases in absenteeism, greater flexibility for women who juggle the demands of work, home and children, and the increased sense of individual dignity that the employees get from having a greater say in organizing their own time.”

Links:  http://embedded.com/columns/breakpoint/206902869

“Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.”

Folk wisdom, as reflected in Brian Kernighan’s clever riff on clever code, is that insight we all know but generally ignore. But how can we measure “cleverness?” Supreme Court Justice Potter Stewart, trying to characterize pornography, memorably lamented that it somewhat defies definition, but “I know it when I see it.” Likewise, most of us can sense excessively clever code but may disagree when examining specific code chunks.

We do know complicated functions are tough to understand, an agony to debug, and usually require excessive amounts of maintenance.

One of the tragedies of software engineering is that, in industry at least, we’ve successfully avoided the metrics-based quality revolution that transformed manufacturing. Few companies know their pretest bug rates in, say, defects per line of code. Or productivity in lines of code per hour. Or which are the worst functions in a project. In an experiment I conducted surreptitiously last year, only a third of correspondents could answer the seemingly simple question “how big is your code” in any metric they wanted to use (MB, lines of code, lines they developed, value in dollars, or, heck, the weight of the listings).

A management mantra states “if you can’t measure it, you can’t manage it.” Though surely true for building widgets, it’s a lot harder to apply to engineering. But some objective measures provide valuable insight into the art of developing firmware. Since we know complex functions are problematic, it seems logical that we measure and control complexity in functions.

And it turns out that’s pretty easy to do.

But first, there are two kinds of complexity: essential, which is a quality about the problem being solved, and incidental, which is the complexity introduced by our engineering approaches. Here we’re talking about incidental complexity, as that’s the only kind we as developers can manage.

Thirty-one years ago, Thomas McCabe proposed a metric called “cyclomatic complexity” to attach a quantitative measure to a function’s complexity. It’s one of many such measures but is by far the one most used. It measures the number of “edges” and “nodes” in a function, where a node is a computational statement and an edge represents the transfer of control between nodes. Cyclomatic complexity is then:

v(G)=edges–nodes+2

Links:  http://embedded.com/columns/technicalinsights/206901032

Many flaws are exploitable because the address space of the vulnerable program has memory that is both writable and executable. As a result, an attacker can write to memory using a buffer overflow, for example, and then transfer control to the address of the injected code.

This problem can be addressed by ensuring that memory pages have the minimal permissions required for proper operation (an application of the principle of least privilege). This is not a comprehensive defense against all exploits, but simply a mechanism designed to extend your defense-in-depth strategy by increasing the difficulty for an attacker to exploit a vulnerability.

Several existing systems enforce reduced privileges in the kernel so that no part of the process address space is both writable and executable. This policy has been named “W xor X” or more concisely WAX. Enforcement of this policy results, for example, in a nonexecutable stack.

OpenBSD is an example of a system that implements a WAX policy. In OpenBSD, an application can still request explicit permissions with mprotect() to override the default.

The implementation of a WnX policy depends on the central processing unit (CPU) and memory management unit (MMU) architecture. For processors that feature an execute bit for each page of memory; OpenBSD implements two changes to make this possible:

1. The signal trampoline is removed from the stack. The signal trampoline page is given read and execute permissions, while the stack segment is given read and write permissions. By doing this, the stack becomes nonexecutable.

2. The mapping of shared libraries and heap memory space is changed so that the data segments do not contain objects which are read, write, and executable. This entails moving the shared library global offset table (GOT), the shared library procedure linkage table (PLT), C++ constructors (.ctors), and C++ destructors (.dtors).

The GOT and PLT are mapped onto their own pages, which are made nonwritable. Minor changes to the dynamic linker are needed to accommodate this change. The . dtors and . ctors sections are moved in with the GOT and consequently become nonwritable as well. Making these changes eliminates executable objects from the data segment.

The implementation for architectures that do not support a per-page execute bit (such as IA-32) is more involved but achieves a similar result.

To supplement this scheme, OpenBSD also reduces permissions on readonly strings and pointers stored as constant data. Historically, these objects were stored in the text segment and had permissions of PROT READ IPROT EXEC. This meant that constant data could potentially be executed as shell code.

Because OpenBSD uses the ELF object file format, a new segment, aptly named . rodata, was created to store this data. As a result, this behavior was changed so that now these objects only have PROT READ and they lose PROT EXEC.

Links: http://embedded.com/columns/technicalinsights/202802744

LILO is the most used Linux Loader for the x86 flavor of Linux; I’ll call it Lilo rather than LILO here because I don’t appreciate uppercase. This file describes some typical Lilo installations. It’s intended as a supplement to the Lilo User’s Guide. I think examples are informative even if your setup isn’t much like mine. I hope this saves you trouble. Since Lilo’s own documentation is very good, who’s interested in the details is referred to /usr/doc/lilo* (once upon a time said gentlemen like Cameron Spitzer and Alessandro Rubini who have made early versions of this document)

LILO (LInux LOader) is a generic boot loader for Linux.

LILO was originally developed by Werner Almesberger, while its current developer is John Coffman.

LILO does not depend on a specific file system, and can boot an operating system (e.g., Linux kernel images) from floppy disks and hard disks. One of up to sixteen different images can be selected at boot time. Various parameters, such as the root device, can be set independently for each kernel. LILO can be placed either in the master boot record (MBR) or the boot sector of a partition. In the latter case something else must be placed in the MBR to load LILO.

At system start, only the BIOS drivers are available for LILO to access hard disks. For this reason, with very old BIOSes, the accessible area is limited to cylinders 0 to 1023 of the first two hard disks. For later BIOSes, LILO can use 32-bit “logical block addressing” (LBA) to access practically the entire storage of all the harddisks that the BIOS allows access to.

LILO was the default bootloader for most Linux distributions in the years after the popularity of loadlin. Today, most distributions use GRUB as the default bootloader.

Links:

http://tldp.org/HOWTO/LILO.html

http://en.wikipedia.org/wiki/LILO_(boot_loader) 

 GRUB is independent of any particular operating system and may be thought of as a tiny, function-specific OS. The purpose of the GRUB kernel is to recognize filesystems and load boot images, and it provides both menu-driven and command-line interfaces to perform these functions. The command-line interface in particular is quite flexible and powerful, with command history and completion features familiar to users of the bash shell.

GRUB is in its element with the multiboot, multidisk systems typical of Linux and open-source adventurers who may simultaneously test or track several Linux distributions, the BSDs, GNU/Hurd, BeOS and perhaps that vestigial partition for Mr. Bill. Even if you stick with LILO as your system’s primary boot loader, it’s smart to keep a GRUB boot floppy handy as the best and fastest way to get your system back if you otherwise cream your master boot record (MBR). If you have done any number of multiboot installations, you know exactly what I’m talking about. Should you need any more reasons for considering GRUB, check out the sidebar, “Why GRUB”. Let’s get started!

 Links:

http://www.linuxjournal.com/article/4622

http://www.gnu.org/software/grub/

Most computer systems can only execute code found in the memory (ROM or RAM). Modern operating systems are stored on hard disks, or occasionally on LiveCDs, USB flash drives, or other non-volatile storage devices. When a computer is first powered on, it does not have an operating system in memory. The computer’s hardware alone cannot perform complex actions such as loading a program from disk, so an apparent paradox exists: to load the operating system into memory, one appears to need to have an operating system already loaded.

The solution is to use a special small program, called a bootstrap loader, bootstrap or boot loader. This program’s only job is to load other software for the operating system to start. Often, multiple-stage boot loaders are used, in which several small programs of increasing complexity sequentially summon one after the other, until the last of them loads the operating system. The name bootstrap loader comes from the image of one pulling oneself up by one’s bootstraps. It derives from the very earliest days of computers and is possibly one of the oldest pieces of computer terminology in common use.

Early computers had a row of toggle switches on the front panel to allow the operator to manually enter the binary boot instructions into memory before transferring control to the CPU. The boot loader would then read the operating system in from an outside storage medium such as paper tape, punched card, or a disk drive.

A boot loader typically consists of three programs:

• The boot sector program is directly loaded by the BIOS at boot time and is only 512 bytes in size.
• The second stage program is loaded by the boot sector program and it does everything you expect the boot loader to do.
• The boot loader installer is not run when the system is booted, but it is used to install the boot loader and the second stage program onto the boot disk. These have to be stored in special locations, so they cannot be copied with cp.

Links:

http://en.wikipedia.org/wiki/Booting 

http://www.xs4all.nl/~lennartb/bootloaders/node3.html

The SYSLINUX Project covers a suite of lightweight bootloaders, for starting up computers in the Linux operating system. It is the work of H. Peter Anvin, and consists of several separate systems.

SYSLINUX is not normally used for booting full Linux installations since Linux is not normally installed on FAT filesystems. Instead, it is often used for boot or rescue floppy discs, Live USBs, or other lightweight boot systems. ISOLINUX is generally used by Linux LiveCD and bootable install CDs.

A minor complication is involved when booting from CD-ROM. The El Torito standard allows for booting in two different modes;

Floppy emulation mode where the boot information is stored in an image file of a floppy disc, which is loaded from the CD and then behaves as a virtual floppy disc. This image file is effectively a FAT filesystem, so SYSLINUX is the required bootloader.

No emulation mode where the boot information is stored directly on the CD (not in a floppy image). In this mode ISOLINUX is required.

To have this choice is sometimes useful however as ISOLINUX is vulnerable to BIOS bugs, in which case it’s handy to be able to boot using SYSLINUX. This mostly affects computers built before about 1999, and, in fact, for modern computers no emulation mode is generally the more reliable method.

Links:

http://en.wikipedia.org/wiki/SYSLINUX

http://syslinux.zytor.com/ 

Almost all embedded systems are event-driven; most of the time they wait for some event such as a time tick, a button press, a mouse click, or the arrival of a data packet. After recognizing the event, the systems react by performing the appropriate computation. This reaction might include manipulating the hardware or generating secondary, “soft” events that trigger other internal software components. Once the event-handling action is complete, such reactive systems enter a dormant state in anticipation of the next event.1

Ironically, most real-time kernels or RTOSes for embedded systems force programmers to model these simple, discrete event reactions using tasks structured as continuous endless loops. To us this seems a serious mismatch–a disparity that’s responsible for much of the familiar complexity of the traditional real-time kernels.

In this article we’ll show how matching the discrete event nature typical of most embedded systems with a simple run-to-completion (RTC) kernel or “tasker” can produce a cleaner, smaller, faster, and more natural execution environment. In fact, we’ll show you how (if you model a task as a discrete, run-to-completion action) you can create a prioritized, fully preemptive, deterministic real-time kernel, which we call Super Simple Tasker (SST), with only a few dozen lines of portable C code.2

Such a real-time kernel is not new; similar kernels are widely used in the industry. Even so, simple RTC schedulers are seldom described in the trade press. We hope that this article provides a convenient reference for those interested in such a lightweight scheduler. But more importantly, we hope to explain why a simple RTC kernel like SST is a perfect match for execution systems built around state machines including those based on advanced UML statecharts. Because state machines universally assume RTC execution semantics, it seems only natural that they should be coupled with a scheduler that expects and exploits the RTC execution model.

We begin with a description of how SST works and explain why it needs only a single stack for all tasks and interrupts. We then contrast this approach with the traditional real-time kernels, which gives us an opportunity to re-examine some basic real-time concepts. Next, we describe a minimal SST implementation in portable ANSI C and back it up with an executable example that you can run on any x86-based PC. We conclude with references to an industrial-strength single-stack kernel combined with an open-source state machine-based framework, which together provide a deterministic execution environment for UML state machines. We’ll assume that you’re familiar with basic real-time concepts, such as interrupt processing, context switches, mutual exclusion and blocking, event queues, and finite state machines.

Links: http://embedded.com/columns/technicalinsights/190302110

Do most embedded projects still need an real time operating system (RTOS)? It’s a good question, given the speed of today’s high performance processors and the availability of patches for real-time Linux, Windows, and other general purpose operating systems (GPOSs). The answer lies in the very nature of embedded devices.

Devices that, in most cases, are manufactured in the thousands, or millions, of units. Devices where even a $1 reduction in per-unit hardware costs can save the manufacturer a small fortune. Devices, in other words, that can’t afford multi-gigahertz processors or a large memory array. In the automotive telematics market, for instance, the typical 32-bit processor runs at about 200Mhz — a far cry from the 2Ghz or faster processors now common in desktops and servers.

In an environment like this, an RTOS designed to extract extremely fast (and predictable) real-time response times from lower-end hardware offers a serious economic advantage. Savings aside, the services provided by an RTOS make many computing problems easier to solve, particularly when multiple activities compete for a system’s resources.

Consider, for instance, a system where users expect (or need) immediate response to input. With an RTOS, a developer can guarantee that operations initiated by the user will execute in preference to other system activities, unless a more important activity (for instance, an operation that helps protect the user’s safety) must execute first.

Consider also a system that must satisfy quality of service (QoS) requirements, such as a device that presents live video. If the device depends on software for any part of its content delivery, it can experience dropped frames at a rate that users perceive as unacceptable – from the users’ perspective, the device is unreliable. But, with an RTOS, the developer can precisely control the order in which software processes execute and thereby ensure that playback occurs at an appropriate and consistent media rate.

RTOSs Aren’t “Fair”
The need for “hard” real time – and for OSs that enable it – remains prevalent in the embedded industry. The question is, what does an RTOS have that a GPOS doesn’t? And how useful are the realtime extensions now available for some GPOSs? Can they provide a reasonable facsimile of RTOS performance?

Let’s begin with task scheduling. In a GPOS, the scheduler typically uses a “fairness” policy to dispatch threads and processes onto the CPU. Such a policy enables the high overall throughput required by desktop and server applications, but offers no assurances that high-priority, time-critical threads will execute in preference to lower-priority threads.

For instance, a GPOS may decay the priority assigned to a high-priority thread, or otherwise dynamically adjust the priority in the interest of fairness to other threads in the system. A high-priority thread can, as a consequence, be preempted by threads of lower priority. In addition, most GPOSs have unbounded dispatch latencies: the more threads in the system, the longer it takes for the GPOS to schedule a thread for execution. Any one of these factors can cause a high-priority thread to miss its deadlines, even on a fast CPU.

In an RTOS, on the other hand, threads execute in order of their priority. If a high-priority thread becomes ready to run, it can, within a small and bounded time interval, take over the CPU from any lower-priority thread that may be executing. Moreover, the high-priority thread can run uninterrupted until it has finished what it needs to do – unless, of course, it is preempted by an even higher-priority thread. This approach, known as priority-based preemptive scheduling, allows high-priority threads to meet their deadlines consistently, no matter how many other threads are competing for CPU time.

Links: http://embedded.com/columns/technicalinsights/193001454