Papers in Systems: Schön on Dynamic Conservatism

I'm part of a "Papers in Systems" reading club, which meets monthly to discuss a chosen paper. This month we read the lecture transcript from Donald Schön's 1970 REITH lecture series, session 2: Dynamic Conservatism (pdf). I found this to be a very enjoyable and insightful read, especially as someone who has frequently tried (and failed!) …

The Global E-Waste Monitor 2020 Report

Further reading

For more on E-Waste, see our Field Atlas entry: The Problem of Electronic Waste.

Abstract

From the website:

The Global E-waste Monitor 2020 provides the most comprehensive overview of the global e-waste challenge, explains how it fits into international efforts to reach the Sustainable Development Goals, and discusses how to create a sustainable society and circular economy. The report provides a national and regional analysis on e-waste quantities and legislative instruments, and makes predictions until 2030. It also encourages decision-makers to increase activities to measure and monitor e-waste using an internationally recognised methodological framework.

Selections from the forward to the report:

Electrical and electronic equipment (EEE) has become an essential part of everyday life. Its availability and widespread use have enabled much of the global population to benefit from higher standards of living. However, the way in which we produce, consume, and dispose of e-waste is unsustainable. Because of the slow adoption of collection and recycling, externalities –such as the consumption of resources, the emission of greenhouse gases, and the release of toxic substances during informal recycling procedures– illustrate the problem to remain within sustainable limits. Consequently, many countries are challenged by the considerable environmental and human health risks of inadequately managed Waste Electrical and Electronic Equipment (WEEE), widely known as e-waste. Even countries with a formal e-waste management system in place are confronted with relatively low collection and recycling rates.

[…]

In 2019, the world generated 53.6 million metric tons (Mt), and only 17.4% of this was officially documented as properly collected and recycled. It grew with 1.8 Mt since 2014, but the total e-waste generation increased by 9.2 Mt. This indicates that the recycling activities are not keeping pace with the global growth of e-waste.

Besides a global perspective, this report includes national and regional analysis on e-waste quantities and legislative instruments. Although 71% of the world’s population is covered by some form of e-waste policy, legislation, or regulation, greater efforts must be made towards implementation and enforcement in order to encourage the take-up of a collection and recycling infrastructure.

The Global E-waste Monitor 2020 introduces the wider public to the global e-waste challenge, explains how the challenge currently fits into international efforts to reach the SDGs, and discusses how to create a circular economy and sustainable societies. In parallel, we encourage decision-makers to increase their activities to measure and monitor e-waste by using and adopting the internationally recognised methodological framework developed by UNU-SCYCLE, in collaboration with the Partnership on Measuring ICT for Development.

Files

Summary

The report notes that in 2019, the world generated 53.6 million metric tons (Mt) of e-waste, increasing by 9.2 Mt since 2014 and projected to reach 74.7 Mt by 2030. It is notable that this number excludes photovoltaic panels (data on these are not available in the UN Comtrade database). Batteries and automotive components also do not factor into the number, because e-waste systems and schemes do not yet include these items. The actual waste generation is higher than what the report accounts for.

The authors of the report calculated that only 17.4% of this waste stream was officially documented as properly collected and recycled, up 1.8 Mt since 2014. The fate of the other 82.6% is ultimately unknown, but it is estimated that around 8% is directly landfilled or incinerated. 7-20% is moved across national boundaries, whether that is by being refurbished and shipped as second-hand products or being exported illegally under the guise of reuse or as “scrap metal”. The remaining amount is probably mixed with other waste streams, such as plastic and metal waste. While this last category of devices may end up being recycled in the end (e.g. by “backyard recyclers” in developing nations), it is often under inferior conditions without the depollution efforts required by official streams and without full recovery of all valuable metals.

The report is filled with regional information, which is helpful because the state of e-waste management differs widely around the globe. Europe had the highest collection and recycling rate at 42.5%, Asia ranked second at 11.7%, the Americas and Oceania were at 9.4% and 8.8% respectively, and Africa is only at 0.9%. We encourage you to take a look at Chapter 9, Regional E-Waste Key Statistics (pg 68), which will enable you to learn more about the state of e-waste management in your region.

Ultimately, this is a stark look at the situation before us: our recycling activities are falling behind e-waste generation. This has many real impacts, which the report also notes:

  • E-waste contains a number of toxic substances, such as mercury (est. 50 tons annually in undocumented flows), cadmium, lead, brominated flame retardants (BFR) (est. 71 kt annually in undocumented flows), and chlorofluorocarbons (CFCs) or hydrochlorofluorocarbons (HCFCs) (est. 98 Mt of CO2 equivalents were released from undocumented recycling of fridges and air conditioners). When many of these long-lived toxic materials are not properly collected and disposed of, they end up poisoning people and the environment – whether that is through direct exposure to workers or community exposure through food/water/air contamination. These materials damage DNA, impact liver and lung function, lead to male reproductive disorders, reduce sperm quality, trigger hearing loss in children, cause cardiovascular regulatory changes in children, and lead to adverse birth outcomes. It is important that e-waste is recycled in a sound manner.
  • E-waste represents an important and untapped source of secondary raw materials (estimated to be ~25 Mt and $57 billion USD in value) that will become increasingly important as virgin material supply pressures increase.
  • Recycling of e-waste is also a significant opportunity for reducing carbon emissions compared to mining and producing virgin materials. Iron, aluminum, and copper make up the majority of the total e-waste weight. In 2019, the demand for these materials was approximately 39 Mt. Recycling can reduce the number of virgin material production that is required for new electronics, and the report estimates that at the current documented formal recycling rate of 17.4%, there is a potential that recycling helped save up to 15 Mt of CO2 in 2019. We can do much better.

Highlights

  • There is a large potential salvage value in electronic goods (gold, copper, aluminum, rare earths, etc.) if we can figure out how to extract them economically.
  • In 2020, the report authors calculate that 17.4% of electronics were safely recycled. Our mission is to get this to 100% by 2050.
  • a follow-up to the 2017 edition and UNU-SCYCLE’s groundbreaking Global E-waste Monitor 2014. This report shows that the global growth in the generation of e-waste continues.
  • In 2019, the world generated 53.6 million metric tons (Mt), and only 17.4% of this was officially documented as properly collected and recycled. It grew with 1.8 Mt since 2014, but the total e-waste generation increased by 9.2 Mt. This indicates that the recycling activities are not keeping pace with the global growth of e-waste.
  • On average, the total weight (excluding photovoltaic panels) of global EEE consumption increases annually by 2.5 million metric tons (Mt).
  • In 2019, the world generated a striking 53.6 Mt of e-waste (excluding PV panels), an average of 7.3 kg per capita. The global generation of e-waste grew by 9.2 Mt since 2014 and is projected to grow to 74.7 Mt by 2030 – almost doubling in only 16 years. The growing amount of e-waste is mainly fueled by higher consumption rates of EEE, short life cycles, and few repair options. Asia generated the highest quantity of e-waste in 2019 at 24.9 Mt, followed by the Americas (13.1 Mt) and Europe (12 Mt), while Africa and Oceania generated 2.9 Mt and 0.7 Mt, respectively. Europe ranked first worldwide in terms of e-waste generation per capita, with 16.2 kg per capita. Oceania was second (16.1 kg per capita), followed by the Americas (13.3 kg per capita), while Asia and Africa generated just 5.6 and 2.5 kg per capita, respectively.
  • In 2019, the formal documented collection and recycling was 9.3 Mt, thus 17.4% compared to e-waste generated. It grew with 1.8 Mt since 2014, an annual growth of almost 0.4 Mt. However, the total e-waste generation increased by 9.2 Mt, with an annual growth of almost 2 Mt. Thus the recycling activities are not keeping pace with the global growth of e-waste. The statistics show that in 2019, the continent with the highest collection and recycling rate was Europe with 42.5%, Asia ranked second at 11.7%, the Americas and Oceania were similar at 9.4% and 8.8%, respectively, and Africa had the lowest rate at 0.9%.
  • The fate of 82.6% (44.3 Mt) of e-waste generated in 2019 is uncertain, and its whereabouts and the environmental impact varies across the different regions. In high income countries, a waste recycling infrastructure is usually developed, and:
    • Around 8% of the e-waste is discarded in waste bins and subsequently landfilled or incinerated. This is mostly comprised of small equipment and small IT.
    • Discarded products can sometimes still be refurbished and reused, and thus are usually shipped as second-hand products from high-income to low- or middle-income countries. However, a considerable amount of e-waste is still exported illegally or under the guise of being for reuse or pretending to be scrap metal. It can be assumed that the volume of transboundary movements of used EEE or e-waste ranges from 7-20% of the e-waste generated.
    • The majority of undocumented domestic and commercial e-waste is probably mixed with other waste streams, such as plastic waste and metal waste. This means that easily recyclable fractions might be recycled but often under inferior conditions without depollution and without the recovery of all valuable materials. Therefore, such recycling is not preferred.
  • In middle- and low-income countries, the e-waste management infrastructure is not yet fully developed or, in some cases, is entirely absent. Hence, e-waste is managed mostly by the informal sector. In this case, e-waste is often handled under inferior conditions, causing severe health effects to workers as well as to the children who often live, work and play near e-waste management activities.
  • Since 2014, the number of countries that have adopted a national e-waste policy, legislation, or regulation has increased from 61 to 78. However, regulatory advances in some regions are slow, enforcement is poor, and policy, legislation, or regulation does not yet stimulate the collection and proper management of e-waste due to lack of investment and political motivation. In addition, the product scope in the legislation is usually different than the e-waste classification systems suggested by the commonly used, internationally harmonised methodological framework on e-waste statistics. These differences in the product scopes lead to a lack of harmonisation of e-waste statistics across countries.
    • Countries are adopting policies, but they are non-standard and lack harmonization across countries
  • E-waste contains several toxic additives or hazardous substances, such as mercury, brominated flame retardants (BFR), and chlorofluorocarbons (CFCs), or hydrochlorofluorocarbons (HCFCs). The increasing levels of e-waste, low collection rates, and non-environmentally sound disposal and treatment of this waste stream pose significant risks to the environment and to human health. A total of 50 t of mercury and 71 kt of BFR plastics are found in globally undocumented flows of e-waste annually, which is largely released into the environment and impacts the health of the exposed workers.
  • Improper management of e-waste also contributes to global warming. First of all, if the materials in e-waste are not recycled, they cannot substitute primary raw materials and reduce greenhouse gas emissions from extraction and refinement of primary raw materials. Next, the refrigerants that are found in some temperature exchange equipment are greenhouse gases. A total of 98 Mt of CO -equivalents were released into the atmosphere from discarded fridges and air-conditioners that were not managed in an environmentally sound manner. This is approximately 0.3% of global energy-related emissions in 2019 (IEA).
  • E-waste is an ‘urban mine’, as it contains several precious, critical, and other non- critical metals that, if recycled, can be used as secondary materials. The value of raw materials in the global e-waste generated in 2019 is equal to approximately $57 billion USD. Iron, copper, and gold contribute mostly to this value. With the current documented collection and recycling rate of 17.4%, a raw material value of $10 billion USD is recovered in an environmental sound way from e-waste globally, and 4 Mt of raw materials could be made available for recycling. The recycling of iron, aluminium, and copper contributed to a net saving of 15 Mt of CO2, equivalent to emissions from the recycling of secondary raw materials substituted to virgin materials.
  • In summary, it is essential to substantially increase the officially documented 17.4% global e-waste collection and recycling rate, especially in view of the rapid growth of this waste stream, which is already projected to reach 74.7 Mt by 2030, combined with increasing recovery of materials towards closed material loops and reducing the use of virgin materials.

What is E-Waste?

  • EEE includes a wide range of products with circuity or electrical components with a power or battery supply (Step Initiative 2014)
  • EEE becomes e-waste once it has been discarded by its owner as waste without the intent of reuse (Step Initiative 2014). Each product has different material content, is disposed of and recycled in different ways, and is unequally harmful to the environment and human health if not managed in an environmentally sound manner.
  • 6 major categories:
    1. temperature exchange equipment (fridges, freezers, heat pumps, ACs)
    2. screens and monitors
    3. lamps
    4. large equipment 9washing machines, dryers, stoves, printing machines, PV panels)
    5. Small equipment (vacuums, microwaves, toasters, kettles, vido cameras, toys, tools, medical devices, monitoring/contorl instruments)
    6. Small IT and telecommunication equipment (phones, GPS devices, calculators, routers, PCs, printers, phones)
  • What’s not accounted for: E-waste systems and schemes do not yet cover any kind of batteries, accumulators, or electrical components of vehicles.

E-Waste Makeup

  • The global quantity of e-waste in 2019 is mainly comprised of Small equipment (17.4 Mt), Large equipment (13.1 Mt), and Temperature exchange equipment (10.8 Mt). Screens and monitors, Small IT and telecommunication equipment, and Lamps represent a smaller share of the e-waste generated in 2019: 6.7 Mt, 4.7 Mt, and 0.9 Mt, respectively.
  • Since 2014, the e-waste categories that have been increasing the most (in terms of total weight of e-waste generated) are the Temperature exchange equipment (with an annual average of 7%), Large equipment (+5%), and Lamps and Small equipment (+4%). This trend is driven by the growing consumption of these products in lower income countries, where the products enhance living standards.
  • Small IT and telecommunication equipment have been growing at lower speed, and Screens and monitors have shown a slight decrease (-1%). This decline can be explained by the fact that, lately, heavy CRT monitors and screens have been replaced by lighter flat panel displays, resulting in a decrease of the total weight even as the number of pieces continue to grow.

E-Waste Policy

  • As of October 2019, 71% of the world’s population was covered by a national e-waste policy, legislation, or regulation. Improvements have been made since 2014 when only 44% of the population was covered. The high coverage rate is affected by the fact that the most populous countries, such as China and India, have national legal instruments in place. However, this population coverage equates to only 78 of the 193 countries. Thus, less than half of all countries in the world are currently covered by a policy, legislation, or regulation.

Development Goals

  • Goal 11: Make cities and human settlements inclusive, safe, resilient, and sustainable
    • Target 11.6: By 2030, reduce the adverse per capita environmental impact of cities by paying special attention to air quality as well as municipal and other waste management. Since over half of the world’s population lives in cities, rapid urbanization requires new solutions to address rising environmental and human health risks, especially in densely populated areas. Most e-waste will be generated in cities, and it is particularly important to properly manage e-waste in urban areas, improve collection and recycling rates, and reduce the amount of e-waste that ends up in dumpsites. The move towards smart cities and the use of ICTs for waste management offer new and exciting opportunities.
    • Indicator 11.6.1: Percentage of urban solid waste regularly collected and with adequate final discharge with regard to the total waste generated by the city.
  • Goal 12: Ensure sustainable consumption and production patterns
    • Target 12.4: By 2030, achieve the environmentally sound management of chemicals and all waste throughout the life cycle, in accordance with agreed-upon international frameworks, and significantly reduce their release into air, water, and soil in order to minimize their adverse impacts on human health and the environment.
    • Indicator 12.4.2: Treatment of waste, generation of hazardous waste, and hazardous waste management, by type of treatment.
    • Target 12.5: By 2030, substantially reduce waste generation through prevention, reduction, repair, recycling, and reuse.
    • An increasing number of people on the planet are consuming growing amounts of goods, and it is critical to make production and consumption more sustainable by raising awareness levels of producers and consumers, specifically in the area of electrical and electronic equipment.

Circular Economy

  • Within the paradigm of a circular economy, the mine of e-waste should be considered an important source of secondary raw materials. Due to issues relating to primary mining, market price fluctuations, material scarcity, availability, and access to resources, it has become necessary to improve the mining of secondary resources and reduce the pressure on virgin materials. By recycling e-waste, countries could at least mitigate their material demand in a secure and sustainable way.
  • On the other hand, the recycling sector is often confronted with high costs of recycling and challenges in recycling the materials. For instance, the recovery of some materials such as germanium and indium is challenging because of their dispersed use in products, and the products are neither designed nor assembled with recycling principles having been taken into account.
  • On the other hand, base metals (e.g. gold) used in certain devices, such as mobile phones and PCs, have a relatively high level of concentration: 280 grams per ton of e-waste. Methods employed to separate and recycle e-waste can be economically viable, especially if carried out manually. Separate collection and recycling of e-waste can thus be economically viable for products containing high concentrations and contents of precious metals. Nevertheless, the recycling rate of most CR is very low and can be improved for precious metals by better collection and pre-treatment of e-waste.
  • Overall, the value of selected raw materials(8) contained in e-waste in 2019 was equal to approximately $57 billion USD(9), corresponding to a total of 25 Mt. Iron, aluminium, and copper represent the majority of the total weight of raw waste materials that can be found in e-waste in 2019. These quantities and the material value could be recovered only in an ideal scenario in which all e-waste generated globally is recycled and the recycling of all selected raw materials is economically viable or even feasible with the recycling technologies currently available.
  • The demand of iron, aluminium, and copper for the production of new electronics in 2019 was approximately 39 Mt. Even in an ideal scenario in which all the iron, copper and aluminium resulting from e-waste (25 Mt) is recycled, the world would still require approximately 14 Mt of iron, aluminium and copper from primary resources to manufacture new electronics (11.6 Mt, 1.4 Mt, and 0.8 Mt, respectively).(10) This indicates that the gap between the secondary iron, aluminium and copper found in e-waste and their demand for the production of new EEE is quite large. This is a consequence of the continuous growth of sales of EEE.
  • With the current documented formal collection and recycling rate of 17.4%, a potential raw material value of $10 billion USD can be recovered from e-waste, and 4 Mt of secondary raw materials would become available for recycling. Focusing only on iron, aluminium, and copper and comparing emissions resulting from their use as virgin raw materials or secondary raw materials, their recycling has helped save up to 15 Mt of CO2 equivalent emissions in 2019 (see Annex 2 for details on the methodology).

Hazards

  • EEE also contains hazardous substances, usually heavy metal such as mercury, cadmium, or lead and chemicals such as chlorofluorocarbons (CFCs), hydrochlorofluorocarbons (HCFCs), and flame retardants. Approximately 71 kt of plastic containing BFR (Brominated Flame Retardants) arise from the unaccounted flows of e-waste generated in 2019 (see Annex 2 for details on the methodology). In particular, BFR are used in appliances to reduce the product’s flammability, appearing, for example, in outer casings of computers, printed wiring boards, connectors, relays, wires, and cables (McPherson, Thorpe, and Blake 2004 & Herat 2008). The recycling of plastic containing BFR represents a major challenge for e-waste recycling because of the costs related to the separation of plastic containing PBDEs and PBBs from other plastic. Recycled plastic with PBDE and PBB content higher than 0.1% cannot be used for manufacturing of any products, including EEEs. In most cases, compliant recyclers incinereate plastic containing PBDEs and PBBs under controlled conditions to avoid the release of dioxins and furans. On the other end, if incineration is not carried out in an environmentally sound manner, those substances are likely to pose risks to health or the environment. The use of PBDEs and PBBs have been banned in Europe (European Parliament 2011). Some of these contaminants have been banned in Europe, as risk assessment studies have shown that they are persistent, bioaccumulative, and toxic, and can be responsible for kidney damage, several skin disorders, and nervous and immune systems and effects to the nervous and immune systems.
  • Mercury is used in fluorescent light sources, e.g. in background lights of older flat panel displays and TVs, in compact fluorescent lamps (“energy-saving lamps”), fluorescent lamps, in measure and control equipment, and in old switches. (Baldé et al. 2018). If these appliances are abandoned in open dumpsites as opposed to being properly recycled, mercury can enter the food chain and accumulate in living organisms while bringing damage to the central nervous system, thyroid, kidneys, lungs, immune system, etc (Baldé et al. 2018). A total of 50 t of mercury can be found in the unaccounted flows of e-waste generated in 2019 worldwide.
  • Chlorofluorocarbons (CFCs) and Hydrochlorofluorocarbons (HCFCs) are present in refrigerant circuits and insulating foams of older generations of cooling and freezing equipment, such as refrigerators, freezers, and air-conditioning systems. These molecules have a long lfespan in the atmosphere. They react with ozone molecules (O3), generating molecular oxygen that thins the stratospheric ozone layer (ozone hole). This process leads to an increment of the UV radiation that can pass the stratosphere, likely causing skin cancers, eye-related diseases, and a weakening of the immune system. The Montreal Protocol (adopted in 1987) regulates the production and consumption of manmade chemicals known as ozone-depleting substances, which includes the phasing out of CFCs and HCFCs. These gases have high global warming potential (GWP). If EEE containing these gases is not managed in an environmentally sound manner, refrigerants could be emitted into the atmosphere. Estimations show that a total of 98 Mt of CO2 equivalents(11) were released from the inferior recycling of undocumented fridges and air conditioners (40% in Europe and 82.6% in the rest of the world). Greenhouse gas (GHG) emissions from the improperly managed refrigerants estimated to be found in air conditioners overtook the emissions from fridges in 2013. In 2019, of the total CO2 equivalents estimated to be released into the atmosphere, 73% were from air conditioners and 27% were from fridges. This is explained by the fact that refrigerants with high global warming potential were used before 1994 (e.g. R-11 and R-12) and until 2017 (R-134a and R-22). Since then, the refrigerants have been substituted by others with a substantially lower GWP (e.g. R-152a and R-124yf). The decrease of CO2 equivalent emissions, reflecting the recent obligations for replacing the refrigerants, will be observed only in the next decades, when the new products placed on the market will become waste (see Annex 2 for details on the methodology).
  • The presence of hazardous substances and scarce or valuable materials in e-waste makes it necessary to recycle and treat the e-waste in an environmentally sound manner; doing so helps avoid the release of such substances into the environment and the losses of ecologically and economically valuable materials. Although several pieces of legislation have banned the use of some substances and are pushing for them to be replaced by safer materials, appliances that were produced in the past and still contain those substances must, once discarded, be treated adequately in order to contain the risks that they can pose to the environment and health. In addition, new equipment may also still contain smaller amounts of those banned substances, due to the fact that they technically cannot yet be substituted or eliminated.

Regional Information

  • Good look at different regions – recommend that you take a look at the report and read about your own global region
  • The United States of America does not have national legislation on the management of e-waste, but 25 states and the District of Columbia have enacted some form of legislation. The state laws vary in their scope and impact and in whether or not they prohibit consumers from disposing electronics in landfills. In all, the laws cover 75-80% of the USA population. However, due to the differences in scope, many areas of the country, including states covered by laws, do not have convenient collection opportunities. Apart from California and Utah, all states that have implemented laws use an EPR approach. Canada does not have a national legislation in effect on the management of e-waste, as the federal agency would not have this authority. However, 12 provinces and territories have regulations in place with industry-managed programmes – all but Nunavut, the least populated territory in Canada. On average, the product scope is much wider than USA; in many Canadian provinces, the EPR requirements can be met by joining an approved e-waste compliance scheme.
  • The USA undertook general measures to prevent e-waste at the federal level and, so, does have a set of regulatory measures for limiting the adverse effects posed by unappropriated disposal and treatment of electronics. Certain electronics, if meeting certain criteria, must be managed under the requirements of the Resource Conservation and Recovery Act (RCRA). Federal agencies are directed to use electronics recyclers that are certified according to either the Responsible Recycling (R2) or e-Stewards standards. Hundreds of electronics recycling facilities have been independently certified to one or both of the certification programmes, whose standard have been updated and enhanced since their inception in 2010.
  • In Europe, the majority of e-waste is regulated by the WEEE Directive (2012/19/EU). This regulation is in force in the European Union and in Norway. Other countries – including Iceland, Switzerland, and several Balkan countries, such as Serbia and Bosnia and Herzegovina – have similar laws. The WEEE Directive set collection, recycling, reuse, and recovery targets for all six categories of e-waste. From 2018 onwards, article 7 of the WEEE Directive states that the minimum collection rate to be achieved annually by a member state shall be either 65% of the average weight of EEE POM in the three preceding years or 85% of e-waste generated on the territory of a member state in 2018. Bulgaria, the Czech Republic, Latvia, Lithuania, Hungary, Malta, Poland, Romania, Slovenia, and Slovakia may have the option to remove themselves from this regulation by 2021 because of their relatively low level of EEE consumption. The latest developments in the implementation of the WEEE Directive are the introduction of the open scope and newly specified reporting guidelines.

WEF State of the Connected World, 2023 Edition

7 September 2023 by Phillip JohnstonThis is a report put out by the World Economic Forum. The report was actually an interesting read, though I focused on the most interesting details for technical teams in the summary below. Abstract As the world begins to emerge from the COVID-19 pandemic, technological advances, such as the internet of things (IoT) and related technologies, have offered an exceptional opportunity to help build a more prosperous and sustainable future. The pandemic has emphasized the importance of IoT and related technologies in people’s lives and work; from contact tracing to wearable devices, these technologies provide …

To access this content, you must purchase a Membership - check out the different options here. If you're a member, log in.

PSA Certified 2023 Report

7 September 2023 by Phillip JohnstonAbstract The PSA Certified 2023 Security Report, gathered from a survey of 1,240 technology decision makers, reveals that security certification is now an essential part of the customer purchasing decision. Security investment costs are increasing as a result: the advantage belongs to those who can comply with regulation and set the conversation. Files PDF Report Reading Club Discussion This paper was selected for our members’ reading club. Follow this link to discuss the paper. Summary I didn’t find as much interesting insights in this survey as I have in others. General impressions are useful, however. …

To access this content, you must purchase a Membership - check out the different options here. If you're a member, log in.

MITRE’s 2023 List of Top 25 Most Dangerous Software Weaknesses

31 August 2023 by Phillip JohnstonBroadly speaking, there are two ways to improve at anything: acquiring new knowledge/skills and reducing errors. The former tends to be prioritized, as many mentally equate proficiency and mastery with a a broad repertoire of skills. But after a certain point, you are hindered much more by weaknesses and errors than a lack of skill or knowledge. You can make significant gains by addressing your weaknesses and eliminating errors. This aspect of improving by eliminating errors stands out to us after reviewing MITRE’s 2023 list of the Top 25 Most Dangerous Software Weaknesses. This is …

To access this content, you must purchase a Membership - check out the different options here. If you're a member, log in.

Architecture Anti-Patterns: Automatically Detectable Violations of Design Principles

17 August 2023 by Phillip Johnston • Last updated 22 August 2024I thoroughly enjoyed Architecture Anti-Patterns: Automatically Detectable Violations of Design Principles, by Mo, Cai, Kazman, Xiao, and Feng. It’s worth reading in its entirety if you are interested in their methods, but you could also just get value from reviewing the description of the architectural anti-patterns or reading the first 2 pages (which I further summarize below). Abstract In large-scale software systems, error-prone or change-prone files rarely stand alone. They are typically architecturally connected and their connections usually exhibit architecture problems causing the propagation of error-proneness or change-proneness. In this paper, we …

To access this content, you must purchase a Membership - check out the different options here. If you're a member, log in.

Programming as Theory Building

Programming as Theory Building  is a classic paper by Peter Naur. After reading the paper, I can see why it has been oft recommended and has maintained staying power all these decades. The situation that Naur describes is just as accurate today.

Abstract

Here’s the intro to the PDF version:

In the article, which follows, note that the quality of the designing programmer’s work is related to the quality of the match between his theory of the problem and his theory of the solution. Note that the quality of a later programmer’s work is related to the match between his theories and the previous programmer’s theories.

Using Naur’s ideas, the designer’s job is not to pass along “the design” but to pass along “the theories” driving the design. The latter goal is more useful and more appropriate. It also highlights that knowledge of the theory is tacit in the owning, and so passing along the theory requires passing along both explicit and tacit knowledge.

Files

Reading Club Discussion

This paper was selected for our members’ reading club. Follow this link to discuss the paper.

Summary

Naur’s hypothesis is that programming, which involves both the design and the actual coding of a software system, is more akin to an embodied skill like playing music or painting or writing. You can’t just blindly follow rules to produce excellent works. You need to learn, ideally with a teacher who can give you feedback, and you must apply a model/concept/theory that you develop in your head to your work.

Applied to software development, Naur points out that it takes more than the source code and documentation to be able to work on a system. You need to develop and maintain a cohesive theory of how the system works. This is developed among the team during the initial development and ongoing maintenance. This theory is rarely captured sufficiently in the documentation or the source code, and thus it cannot be easily created without individuals who maintain the theory in their minds. If you’re a new person and want to effectively work on the system, you must be able to interact with other developers who have the theory of operations in their mind. They will be able to answer your questions and to provide feedback in why (or why not) particular approaches are suitable for the system.

What matters most, in Naur’s view is the theory of the system – not the code or the documentation. The system theory is what allows the programmer to explain what the system does and effectively make changes. The challenge is that this theory largely lives in the heads of the programmers. It is more than just “rules” to follow. It is embodied knowledge.

By extension, it matters that each member of the team has a shared theory of the system. The longer that a shared conceptual view can be maintained, the better the system can be maintained. If people have different theories, they will inevitably clash with the changes they make, until the system has decays into a mess. Maintaining system quality and integrity is thus a problem of maintaining a cohesive mental model of the system and its operations. One way we can develop better maintain cohesion is by developing metaphors for the system (or its parts) that can be easily shared and understood by all who are working on the system.

Key Lessons

There are a couple of critical points that Naur makes at the end of the paper, and we should keep these in mind:

  • The death of a program happens when the team that has the system’s theory in their heads dissolves
  • Life of a program is extended by successfully passing on the theory to a new generation of programmers
  • Reviving a program after the team has dissolved is essentially impossible – you probably can’t recreate the original theory in your own mind, and a rewrite would be more effective as you’re actually developing a cohesive mental model of your own that can then be passed on. 
  • If you’re documenting a system from the development/maintenance perspective, what matters most is that you document the operational theory, connections between things, and metaphors for different parts of the system.

My Thoughts

I’m sure that many of us have felt this about code, even if we have not conceived of it it in this way before. All you have to do is look at someone else’s code: what makes perfect sense to them can be completely unapproachable to you. Or maybe you submit a patch to an open source project, getting it rejected for some obscure reason you couldn’t predict just be looking at the project. Or join a new team and struggle to get up-to-speed with the system. Or get handed a legacy project where the original developer has long since moved on, and try to make sense out of the mess. 

I feel the reality of this model deeply with my forays in to the Medtronic PB560 source code. I had thought I would have made much more progress on the review than I have. But progress is slow: I cannot understand why the system works the way it does, how different pieces interact, and what the overall theory is just by looking at the source code. If I could run the code, it would be a different story – I could develop my own theory through experimentation, even if it wasn’t a match to the original developers.

I also think Naur’s idea explains the prevalence of “Not invented here” syndrome: it’s “easier” to rewrite something and develop your own conceptual model than it is to pick up someone else’s. Of course, being justifiable does not mean that it is the rational choice. I work with plenty of libraries for which I do not need to have a conceptual model of the implementation, because I am not maintaining it.

Highlights

You can see the annotated text here.

suggests that programming properly should be regarded as an activity by which the programmers form or achieve a certain kind of insight, a theory, of the matters at hand. This suggestion is in contrast to what appears to be a more common notion, that programming should be regarded as a production of a program and certain other texts.

I like the inclusion of design along with implementation, which tends to be the exclusive focus of “programming”:

shall use the word programming to denote the whole activity of design and implementation of programmed solutions.

Examples given by Naur: access to the team makes all the difference, even if you have full source code and documentation.

In the present context the significant issue is the importance of the personal advice from group A in the matters that concerned how to implement the extensions M to the language. During the design phase group B made suggestions for the manner in which the extensions should be accommodated and submitted them to group A for review. In several major cases it turned out that the solutions suggested by group B were found by group A to make no use of the facilities that were not only inherent in the structure of the existing compiler but were discussed at length in its documentation, and to be based instead on additions to that structure in the form of patches that effectively destroyed its power and simplicity. The members of group A were able to spot these cases instantly and could propose simple and effective solutions, framed entirely within the existing structure.

[…]

This is an example of how the full program text and additional documentation is insufficient in conveying to even the highly motivated group B the deeper insight into the design, that theory which is immediately present to the members of group A.

[…]

In the years following these events the compiler developed by group B was taken over by other programmers of the same organization, without guidance from group A. Information obtained by a member of group A about the compiler resulting from the further modification of it after about 10 years made it clear that at that later stage the original powerful structure was still visible, but made entirely ineffective by amorphous additions of many different kinds. Thus, again, the program text and its documentation has proved insufficient as a carrier of some of the most important design ideas.

The conclusion seems inescapable that at least with certain kinds of large programs, the continued adaption, modification, and correction of errors in them, is essentially dependent on a certain kind of knowledge possessed by a group of programmers who are closely and continuously connected with them.

Very briefly, a person who has or possesses a theory in this sense knows how to do certain things and in addition can support the actual doing with explanations, justifications, and answers to queries, about the activity of concern.

In intelligent behaviour the person displays, not any particular knowledge of facts, but the ability to do certain things, such as to make and appreciate jokes, to talk grammatically, or to fish. More particularly, the intelligent performance is characterized in part by the person’s doing them well, according to certain criteria, but further displays the person’s ability to apply the criteria so as to detect and correct lapses, to learn from the examples of others, and so forth. It may be noted that this notion of intelligence does not rely on any notion that the intelligent behaviour depends on the person’s following or adhering to rules, prescriptions, or methods.

In terms of Ryle’s notion of theory, what has to be built by the programmer is a theory of how certain affairs of the world will be handled by, or supported by, a computer program. On the Theory Building View of programming the theory built by the programmers has primacy over such other products as program texts, user documentation, and additional documentation such as specifications.

the programmer’s knowledge transcends that given in documentation in at least three essential areas:

  1. The programmer having the theory of the program can explain how the solution relates to the affairs of the world that it helps to handle. […]
  2. The programmer having the theory of the program can explain why each part of the program is what it is, in other words is able to support the actual program text with a justification of some sort. […]
  3. The programmer having the theory of the program is able to respond constructively to any demand for a modification of the program so as to support the affairs of the world in a new manner. Designing how a modification is best incorporated into an established program depends on the perception of the similarity of the new demand with the operational facilities already built into the program.

One thing seems to be agreed by everyone, that software will be modified. It is invariably the case that a program, once in operation, will be felt to be only part of the answer to the problems at hand. Also the very use of the program itself will inspire ideas for further useful services that the program ought to provide. Hence the need for ways to handle modifications.

The question of program modifications is closely tied to that of programming costs. In the face of a need for a changed manner of operation of the program, one hopes to achieve a saving of costs by making modifications of an existing program text, rather than by writing an entirely new program.

Naur provides some counterpoints to my ideas around design for change. But I have to note that “low cost” (his terms) are different from the goal of reducing the cost. You can have reduced costs that are still high.

The expectation that program modifications at low cost ought to be possible is one that calls for closer analysis. First it should be noted that such an expectation cannot be supported by analogy with modifications of other complicated man–made constructions. Where modifications are occasionally put into action, for example in the case of buildings, they are well known to be expensive and in fact complete demolition of the existing building followed by new construction is often found to be preferable economically. Second, the expectation of the possibility of low cost program modifications conceivably finds support in the fact that a program is a text held in a medium allowing for easy editing. For this support to be valid it must clearly be assumed that the dominating cost is one of text manipulation. This would agree with a notion of programming as text production. On the Theory Building View this whole argument is false. This view gives no support to an expectation that program modifications at low cost are generally possible.

A further closely related issue is that of program flexibility. In including flexibility in a program we build into the program certain operational facilities that are not immediately demanded, but which are likely to turn out to be useful. Thus a flexible program is able to handle certain classes of changes of external circumstances without being modified.

On this point below I just totally disagree that it’s the goal, and sort of a strawman argument. There’s a difference between “easier to change” and “flexible enough to handle all foreseeable possible needs”. And the costs here are completely imagined according to the scenario. For example, is it really so expensive for me to abstract away the underlying SPI controller with a read/write API that needs to be supplied from the outside? Absolutely not.

It could be that we view “change” differently enough to make this conclusion obvious to him, but specious to me.

It is often stated that programs should be designed to include a lot of flexibility, so as to be readily adaptable to changing circumstances. Such advice may be reasonable as far as flexibility that can be easily achieved is concerned. However, flexibility can in general only be achieved at a substantial cost. Each item of it has to be designed, including what circumstances it has to cover and by what kind of parameters it should be controlled. Then it has to be implemented, tested, and described. This cost is incurred in achieving a program feature whose usefulness depends entirely on future events. It must be obvious that built–in program flexibility is no answer to the general demand for adapting programs to the changing circumstances of the world.

Now, on this point below, we totally agree. And perhaps this ties better into DfC points anyway: that when we make changes blindly or following the “most obvious route” without care to maintaining a theory of operations or design, then we get a tangled mess that is increasingly difficult to change.

What stands out at me is Naur’s point: “For a program to retain its quality it is mandatory that each modification is firmly grounded in the theory of it.” How easy it is to see when modifications clash with the theory!

On the basis of the Theory Building View the decay of a program text as a result of modifications made by programmers without a proper grasp of the underlying theory becomes understandable. As a matter of fact, if viewed merely as a change of the program text and of the external behaviour of the execution, a given desired modification may usually be realized in many different ways, all correct. At the same time, if viewed in relation to the theory of the program these ways may look very different, some of them perhaps conforming to that theory or extending it in a natural way, while others may be wholly inconsistent with that theory, perhaps having the character of unintegrated patches on the main part of the program. This difference of character of various changes is one that can only make sense to the programmer who possesses the theory of the program. At the same time the character of changes made in a program text is vital to the longer term viability of the program. For a program to retain its quality it is mandatory that each modification is firmly grounded in the theory of it. Indeed, the very notion of qualities such as simplicity and good structure can only be understood in terms of the theory of the program, since they characterize the actual program text in relation to such program texts that might have been written to achieve the same execution behaviour, but which exist only as possibilities in the programmer’s understanding.

The building of the program is the same as the building of the theory of it by and in the team of programmers. During the program life a programmer team possessing its theory remains in active control of the program, and in particular retains control over all modifications. The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered. Revival of a program is the rebuilding of its theory by a new programmer team.

The extended life of a program according to these notions depends on the taking over by new generations of programmers of the theory of the program. For a new programmer to come to possess an existing theory of a program it is insufficient that he or she has the opportunity to become familiar with the program text and other documentation. What is required is that the new programmer has the opportunity to work in close contact with the programmers who already possess the theory, so as to be able to become familiar with the place of the program in the wider context of the relevant real world situations and so as to acquire the knowledge of how the program works and how unusual program reactions and program modifications are handled within the program theory.

This problem of education of new programmers in an existing theory of a program is quite similar to that of the educational problem of other activities where the knowledge of how to do certain things dominates over the knowledge that certain things are the case, such as writing and playing a music instrument. The most important educational activity is the student’s doing the relevant things under suitable supervision and guidance.

I totally think the below is the case from trying to look at the PB 560 code base. It’s hard for me to keep up progress precisely because I get stuck.

A very important consequence of the Theory Building View is that program revival, that is reestablishing the theory of a program merely from the documentation, is strictly impossible.

The complete rewrite is the only hope…

In preference to program revival, the Theory Building View suggests, the existing program text should be discarded and the new–formed programmer team should be given the opportunity to solve the given problem afresh.

2023 FDA Medical Device Security Requirements

10 April 2023 by Phillip JohnstonIncluded in the U.S. 2023 Consolidated Appropriations Act (H.R. 2617) (the “spending bill”) were new cybersecurity regulations for U.S. medical devices. Sec. 3305 of the bill, Ensuring Cybersecurity of Medical Devices, adds a new section to the Federal Food, Drug, and Cosmetic Act: “524B. Ensuring Cybersecurity of Devices”. New Requirements If your medical device can connect to the internet, or contains software that could be vulnerable to cybersecurity threats in some way, your FDA applications will now need to include information that shows your company: Has a plan to monitor, identify, and address, in a …

To access this content, you must purchase a Membership - check out the different options here. If you're a member, log in.

EU Cyber Resilience Act

1 April 2023 by Phillip JohnstonIn September 2022, the European Commission proposed a Cyber Resilience Act (proposal pdf). The call for feedback has ended, and a new draft is expected in 2023. We are still reading the legislation, so updates will continue to be made over the next few days. Table of Contents: Background Objectives Terminology Note – Manufacturers and Production The Explicit Strategy Implementation Enforcement Plans Related References Background The motivation behind this act is a commonly expressed one (and, essentially, the same as outlined by the U.S. government): Hardware and software products are increasingly subject to successful cyberattacks, …

To access this content, you must purchase a Membership - check out the different options here. If you're a member, log in.