IMPROVING PROGRAM DESIGN
ACCOMPANYING REPORT OF
THE NATIONAL PERFORMANCE REVIEW
OFFICE OF THE VICE PRESIDENT
Washington, DC
September 1993
Contents
Executive Summary................................1
The Need for Change..............................5
Program Design Criteria.........................11
Actions.........................................19
Appendices
A.Summary of Actions by Implementation Category......................................25
B. Accompanying Reports of the National
Performance Review...........................27
Implementation Categories
Each action is followed by a number in parentheses that indicates the necessary avenue for effective implementation. Appendix A organizes all actions according to these categories.
(1) Agency heads can do themselves
(2) President, Executive Office of the President, or
Office of Management and Budget can do
(3) Requires legislative action
(4) Good idea, but will require additional work, or
may be better suited for future action
Abbreviations
CBO Congressional Budget Office
CRS Congressional Research Service
FAA Federal Aviation Administration
GAO General Accounting Office
NAPA National Academy of Public Administration
NPR National Performance Review
PMC President's Management Council
OMB Office of Management and Budget
OPM Office of Personnel Management
SSA Social Security Administration
SSI Supplemental Security Income
USDA United States Department of Agriculture
Federal programs are often a story of good intentions gone awry. Consider, for instance, the case of Supplemental Security Income (SSI) recipients who are also eligible for food stamps. Congress, hoping to improve service to, and participation of, those SSI beneficiaries, concluded that the Departments of Agriculture and Health and Human Services should coordinate the delivery of food stamps at Social Security Administration (SSA) offices. Subsequently, however, the General Accounting Office (GAO) found that this design resulted in duplicated efforts and poor service.
Another flawed program design is the federal student loan program which has guaranteed $142 billion in loans, paid $35 billion in interest, and lost $19 billion in defaulted loans since 1965, yet it generates hundreds of millions of dollars in profit every year for private corporations. Poor program design also contributed to lax federal regulation of the savings and loan industry, which will cost taxpayers some $300 billion before the crisis ends. More recently, the Federal Aviation Administration struggled to keep up with rapid growth in airline service because of serious institutional constraints, prompting some experts to recommend transforming it into a government corporation.
These experiences are not isolated. Many federal programs are badly designed, ill-conceived and fatally flawed. Innumerable bureaucratic horror stories publicized by journalists, pundits, politicians, and civil servants have unfairly turned phrases like "good government" into oxymorons. [Endnote 1] GAO has identified 17 federal programs as high-risk activities especially vulnerable to waste, fraud, abuse, and mismanagement.[Endnote 2] The Office of Management and Budget (OMB) currently has 104 programs on its high-risk list.[Endnote 3]
The number and severity of these problems has undermined the public's trust in government. Perhaps not since the beginning of the Great Depression have Americans shown such little confidence in Washington's ability to provide the service that the nation deserves. And, unlike Watergate, Iran-Contra, and other political scandals, the problem is not rooted in the episodic vices and failures of individuals. Instead, Americans are increasingly frustrated by the enduring deficiencies in our government's structures and processes.
Often, government problems are rooted in the poor foundations upon which public programs are built-- ambiguous goals, weak operational concepts, and careless implementation design. Now, both program design and program evaluation are enjoying heightened prominence with enactment of the Government Performance and Results Act of 1993. It requires the executive branch to produce both performance plans and performance reports; these requirements should prompt agencies to foster better program designs and more rigorous program evaluations.
Federal programs come in a wide assortment. Sometimes, the federal government provides cash payments to individuals, such as with Social Security. At other times, it provides benefits only indirectly, such as when it offers tax incentives for builders to construct low-income housing. The particular method of delivery of public service (e.g., direct subsidies, tax incentives, loan guarantees, regulations, etc.) should be tailored to the specific requirements and circumstances of the constituency to be served.
To be sure, political considerations, both in the executive branch and Congress, play a major role in influencing the shape and funding of most federal programs. Nevertheless, these programs also should be rationally formulated and designed. In fact, sound designs that are based on objective criteria could yield programs that are more likely to win administrative and legislative approval.
At present, the federal government lacks any systematic, disciplined approach to program design. In designing programs, officials all too often fail to anticipate the unintended consequences. Program evaluation in OMB and many agencies is less prominent than it once was. Yet we need broadly applicable techniques for designing effective, cost-efficient programs, especially in an era of fiscal austerity. We also should apply program design technology to existing programs as part of program review, evaluation, and redesign processes. In time, the objective criteria that we develop may help us better determine what services the federal government should provide, and what we should discontinue. The following are examples of such objective criteria:
10.Build in cessation provisions.
Whatever criteria are chosen, the federal government needs to develop a formal discipline of program design. The National Performance Review recommends four presidential initiatives.
First, the President should direct the President's Management Council to sponsor the development and publication of a comprehensive handbook that would address the strengths and weaknesses of alternative program designs. It should include a complete review of relevant literature, a compilation of existing federal programs, a discussion of different approaches, an assessment of comparative advantages, analyses of case studies, and model designs for program types.
Second, the Council should designate one or two agencies to test program design capability to determine its value and costs. For these pilot projects, agencies might create modest program design offices--staffed by individuals qualified in program design, management and evaluation--to help senior officials design new programs and assess existing ones.
Third, the Council should take steps to help Congress design better programs. It should work with the Congressional Research Service (CRS) so the CRS will include program design concepts in all of its relevant training materials. In addition, the Council should encourage CRS, GAO, the Congressional Budget Office, and key congressional staff to participate in program design training.
Fourth, the Council should commission program design courses to educate and train current and future policymakers, program designers, and managers. Such education might include short seminars, a longer residential course, the introduction of program design into existing management development programs, and offerings at graduate schools of public administration.
Endnotes
In a speech at a conference in Atlanta some years ago, a federal official told a story about a young attorney working on Capitol Hill who was asked to draft a bill to regulate some aspect of the airline industry. He started by looking up and copying the law that addressed similar concerns with the railroads. Finding some provisions which did not seem applicable to railroads, he did further research and discovered that the railroad bill had been lifted 75 years earlier by another attorney from a law passed early in the 19th century to regulate canals. Justice Oliver Wendell Holmes said that "in the law, an ounce of history is worth a pound of logic.''[Endnote 1] However, in the design of federal programs, unfortunately both are often missing. Consider the following 1981 exchange before a subcommittee of the House Committee on Banking, Finance and Urban Affairs:
Chairman Blanchard: Have you made any conclusions as to where governmental loans are an effective policy instrument and where they are not? Where loan guarantees are an effective policy instrument and where they are not? How they work, in what instances they work well and where they don't, and where they can be effective and where they aren't? Because we are looking at these in relation to other tools that the government has to operate...Have there been any studies on...federal use of direct spending versus loans versus loan guarantees versus the tax code, and then the various types of tax preferences and their effectiveness in different instances?
The witnesses responded that they were unaware of any such literature.[Endnote 2] Earlier the Congressional Research Service observed:
These program differences underline the ways in which the programs have evolved. There have been no overall guidelines or framework within which these programs are created. They have evolved over time to meet particular needs and situations. This generally ad hoc framework has led to a situation where, except for vast generalizations, it is hard to make specific statements concerning the characteristics of federal loan guarantees.[Endnote 3]
Is there a systematic, disciplined approach to program design in the Federal Government? There is not. An extensive literature search and consultation with experts inside and outside government reveal an overwhelming consensus that the lack of attention to program design means that federal activities are less effective and less economical than they should be. Program designs are frequently characterized by insufficient rational analysis and failure to anticipate unintended consequences. Often this can be attributed to the poor foundations upon which the programs were built- ambiguous goals, weak operational concepts and careless implementation design. It is rare to compare programs across agency and department lines in terms of what they do and how they do it. We do not have a methodology with which to compare them.
Some public policy/administration academics have attempted to develop a rational, disciplined approach to program design. In 1985, A Workbook in Program Design for Public Managers noted the absence of any preexisting attempt to provide this discipline and recognized its own limitations, stating ". . . at best, this workbook is but a beginning effort.''[Endnote 4] In 1986, the classic text Politics, Position and Power (Fourth Edition) proposed twelve criteria for evaluating the design and organization of new or existing programs.[Endnote 5] In 1989, the Urban Institute published a primer on program reviews that outlined criteria for analyzing program alternatives.[Endnote 6]
While academic interest in program design seems to be growing, the practical applications have been few. "Over time, legislation has suffered more and more from wishful thinking, sloppy drafting, and woeful disregard for public-sector organizational principles.''[Endnote 7] A congressional effort to establish a program design discipline for a segment of related governmental programs can be found in the 1988 public law which elevated the Veterans Administration to departmental status. The law provided for the establishment of a National Commission on Executive Organization, generally patterned on the first Hoover Commission. Its charter would have required ". . . establish[ment of] criteria for use by the President and Congress in evaluating proposals for government corporations and government-sponsored enterprises and subsequently overseeing their performance.'' [Endnote 8] The new commission could have been activated by presidential initiative; however, it was not.
Rigorous assessments of federal programs and their designs were once an active responsibility of OMB; however, current oversight responsibilities regarding program design are cloudy. In OMB and many agencies, program evaluation as a related discipline is also less prominent now.
Both program design and program evaluation will become more prominent under the new Government Performance and Results Act of 1993. The Act requires performance plans and performance reports from the executive branch; hence, agencies may be prompted to foster sound program designs as well as rigorous program evaluation.
What are the consequences of the absence of a rational program design process? Too often programs are designed without careful consideration of the way that similar designs have worked previously or the potential benefits of alternate designs.[Endnote 9] It is no wonder news media, federal auditors, politicians and taxpayers are critical of a government that too often creates federal programs that do not meet public expectations. A consequence is sagging public confidence in the ability of government to deliver.
For example, flawed program design accounts for poor public service in food stamp application processing for Supplemental Security Income (SSI) applicants and recipients. Congress believed that food stamp program participation would increase, and more streamlined public service would occur, if the Departments of Agriculture and Health and Human Services coordinated the delivery of food stamp services at Social Security Administration (SSA) offices. For SSI clients eligible and willing to apply for food stamps at SSA field offices, SSA staff complete an application and forward it to a state food stamp office for further consideration.[Endnote 10] GAO concluded:
The process for taking food stamp applications at SSA has not worked well...further, the process as designed results in duplication of effort and poor service to clients...the food stamp applications used are unnecessarily complex and lengthy.'' [Endnote 11]
There are many other examples of poor programs that have received public criticism because they fail to meet various design criteria. These include:
--Regulation of Savings and Loan Institutions. The government intended to use federal deposit insurance to subsidize the cost of loans and increase the availability of mortgage money for home buyers. To avoid the misuse of a government guarantee such as deposit insurance, the government must establish an effective and independent supervisory agency. The Federal Home Loan Bank Board lacked this independence from the savings and loan industry that it was supposed to supervise. The ultimate cost of that regulatory failure will amount to some $100 billion.[Endnote 12]
--Reduction of Crime. In attempting to reduce burglaries, the Law Enforcement Assistance Administration tried to increase the effectiveness of limited numbers of law enforcement officials through large investments in sophisticated telecommunications in police cruisers. These efforts were much less effective than Neighborhood Watch programs which greatly increased the number of people participating in crime surveillance at minimal cost.[Endnote 13] The consideration of alternatives in the design of federal programs is all too rare. Consequently the allocation of program resources often is not optimal in accomplishing desired ends.
--Federal Catfish Research. The Senate has noted that catfish research is conducted in at least four agencies: Fish and Wildlife Service (Interior), Animal and Plant Inspection Service (Agriculture), Agriculture Research Service, and Agriculture Cooperation Service. Although a 1988 report by the Inspector General at the Department of Commerce (itself conducting independent catfish studies) recommended that the work be consolidated, it still has not been done.[Endnote 14] Federal catfish research raises a variety of design questions. How compatible are these research efforts with each other? Are they complementary or redundant? What are the aggregate costs for federal catfish research and are they justified by the benefits?
--National Air Traffic Control. The Federal Aviation Administration (FAA) is responsible for airline safety in the United States and administers the nation's air traffic control system. Commentators have repeatedly pointed to problems confronting the FAA as it works to keep up with rapid growth in airline service and cope with federal budget constraints and inflexibility in its institutional structure. Recently the National Airline Commission recommended that the FAA become a government corporation so that it can regain the capacity to carry out its essential functions.[Endnote 15]
What is a program? In this context, a program is "a particular combination of authority, organizations, resources and personnel assembled to achieve specific public purposes.''[Endnote 16] At times, the federal government's role is very direct, such as payment of cash to Social Security beneficiaries. Alternatively the role can be indirect, such as the establishment of tax incentives for the private sector to provide low income housing.
In the field of public administration, two other levels of analysis have been proposed. One analyzes a set of programs or a program cluster aimed at a specific, common target group.[Endnote 17] For example, there are many programs addressing the income maintenance needs of the poor. Rather than considering a single program's design in isolation, the preferred focus is to diagnose program effectiveness as a whole.
A second alternative is to focus on the generic tools of government action, the techniques of social intervention used in particular public programs.[Endnote 18] Reinventing Government identifies 36 such mechanisms, ranging from grants to tax policies.[Endnote 19] In some program designs, more than one tool may be used; therefore, it is helpful to examine how well each tool works, either alone or in combination with others.
Frequently, each of these perspectives merits consideration. For example, consider a category of programs using more than one method of delivery--job training. It can be delivered in many forms including tax policy, grants, loans, loan guarantees, technical assistance, information, referral, seed money, and jaw boning. Whatever combination of delivery tools may apply to a specific public purpose, it would be desirable to understand the programmatic impact of the tools individually and collectively in order to design the best portfolio.
Why do we need an emphasis on program design? Effective and efficient delivery of programs is a major determinant of the public's confidence in its government. Although the approval and funding of most federal programs are inevitably shaped by political considerations (both in Congress and the executive branch), the formulation and design of these programs should be grounded in rational and valid public purposes. Sound designs based on objective criteria can yield programs more likely to win bureaucratic and political approval. A program design discipline that can increase the likelihood of a program's success is especially important in a time of especially austere fiscal constraints. In addition, this program design discipline should be applied to existing programs as part of program review and redesign processes.[Endnote 20] The American public is willing to support programs if it is confident the programs address issues important to them and the solutions have a reasonable probability of success (e.g., health care reform).[Endnote 21]
Whatever the purposes of any public program, its logic and internal consistency can be exposed and analyzed using objective rational criteria. The following is a set of criteria which could be applied systematically when designing or redesigning programs.[Endnote 1] Research and development to further the field certainly will refine and augment this list.
All program designs require clear, explicit identification of the primary beneficiaries and the specific benefits they can expect. Without this foundation, no rational program design methodology can be sustained. Programs designed to solve particular problems should be consistent with the missions of the agencies responsible. However, both the problems addressed and the missions pursued should always be framed in the context of who benefits.
Prioritize Beneficiaries. The public purposes and program goals must be defined in terms of benefits to individuals and/or groups. There are two levels of beneficiaries, primary and secondary, which must be reflected in a program's design.
Primary beneficiaries benefit directly from a program. To illustrate, the Department of Veterans Affairs serves veterans; the programs it administers clearly define categories of veterans who qualify as beneficiaries. Other programs' intended primary beneficiaries are not as clear. Consider government programs allowing removal of trees from federal lands. In this latter case the class of primary beneficiaries may be the loggers--who would otherwise be unemployed--the logging companies, the domestic and foreign timber consumers, or the recreational users who object to despoiling the environment. Secondary beneficiaries are those individuals, groups of individuals, or organizations benefiting indirectly from a program. For example, state governments are the secondary beneficiaries of federal grants for education.
Finally, a class of interested parties, not strictly beneficiaries of a program, are stakeholders, such as taxpayers. The taxpayers deserve the most effective and efficient design possible to maximize the return on their investment in tax dollars.
Validate the Need. It is essential to verify that the need exists and will persist. The program designers must be able to document that the need is more than a short-term aberration. The determination of who benefits and how they benefit should be validated by the intended beneficiaries themselves, which is the essence of customer-driven government. Federal-level involvement must also be justified. The program must avoid addressing obsolete problems or needs that are more effectively solved through state or local initiatives, through the private sector, or by nonprofit organizations.
Determine Program Size and Scope. Program designers must also determine the size and scope of the population to be served so the course of action matches the problem. How many people are affected? What are the eligibility criteria? What are the characteristics of the affected populations (geography, socio-economic factors, etc.)? What level of diversity exists among the populations affected? What is the minimum program size necessary to achieve the results required?
If there is more than one problem to be addressed, the program designers must identify each need and prioritize solutions to avoid unbalanced or skewed designs. Social Security is an example of a program that has well-defined beneficiaries and missions. It is designed to provide income to individuals (and their eligible family members) who are retired or disabled. The Social Security Administration has long viewed its mission as the right check, to the right person, in the correct amount, on time. Clear eligibility criteria enable reliable estimates of client populations and size of benefits.
Inspire Public Confidence. Program design should give the program credibility with the public. There is a common perception that the political process does not always place program priorities on the appropriate beneficiaries. Without strong public confidence, programs are hindered from gaining approval and overcoming resistance during implementation. A federal program should appear to make sense both in intent and by design as a valid commitment of public resources.
2. Define and Evaluate Alternate Methods of Program
Delivery
After clearly defining program goals and beneficiaries, the designers should consider any alternate forms of program delivery. Not all public needs must be satisfied by direct federal delivery of services. Alternatives to direct government service include regulatory requirements, government sponsored enterprises, tax incentives, government guarantees, monitoring and enforcement, etc.
Different program delivery methods can often attain similar ends, although costs and schedules are not likely to be the same. Education is a prime example. Education has been delivered in many forms such as direct cash payments to veterans, categorical grants to states to fund educational initiatives at the local level, block grants, tax exemptions, privatization, etc.Two sources of potential alternatives are the list of 36 governmental mechanisms that were identified by Osborne and Gaebler in Reinventing Government, and the program tools identified by Professor Lester Salamon of Johns Hopkins University.[Endnote 2] Salamon suggests that the major tools of public action should be studied in a well-structured framework:
Given the great proliferation of instruments that the public sector uses, and the likelihood that this proliferation will continue as a result of the privatization movement and the resource constraints under which government is operating, the development of a body of knowledge that can help guide the selection of tools and instruct program managers about the consequences of these choices seems increasingly worthwhile.[Endnote 3]
Review Against Assumptions. Alternatives must be reviewed explicitly against fundamental assumptions. What are the critical success factors identified by the beneficiaries? How has success been measured and how well does each program design address the needs identified? What is each design's ability to serve people, to adapt to emerging technology, and to use existing facilities? What are the possible unintended consequences or adverse effects of each design mechanism? How do they compare to the status quo? Examples of programs which merit review against their initial assumptions include job training, veterans health care, and sundry entitlement programs.
3. Examine Program Compatibility
It is rare that any new program is so independent and autonomous that it enters a domain totally unpopulated by preexisting efforts. Consequently, program design should be mindful of compatibility and complementarity with existing infrastructure or related programs.
Comparative Advantage. What does this program do that is unique? Would another program or program design perform these functions better? What would beneficiaries lose if this program did not exist?
Complementarity. Does the program complement other programs that serve the same beneficiaries? Does the program needlessly duplicate or overlap other programs at the federal, state, or local levels? Does the program foster cooperation or does it create barriers between beneficiaries and administrators? To illustrate, one-stop shopping, which provides multiple services conveniently at one place, contributes to integrated program delivery and is being considered by the Department of Labor for workforce development.[Endnote 4]
Harmony. Is the program consistent with the mission of the agency which would administer it? Does it create conflict in the agency? Is the program more compatible with another agency's mission? Are another agency's programs more interrelated and would that agency therefore be a better fit? How easy is it to add the program to an existing organizational structure? Will it replace one or more existing programs? Does it create or reduce redundancies?
Catalyst. Does the program foster competition or does it encourage monopolies? Does it serve as a catalyst? Does the program design result in the empowerment of people and communities so they have a vested interest in how the program is run? Is there positive leverage with other programs or destructive interference? Does the sum of all programs encourage the right behavior or do perverse incentives motivate dysfunctional behavior?
4. Assess Cost-Effectiveness and Efficiency
Program designers should apply cost-effectiveness and efficiency criteria to the program as a whole and to specific program elements (including organizational structures, program delivery, and administrative support functions) to estimate:
--cost (total, marginal, and/or
administrative, as a measure of overhead) per unit of
benefit (defined as program outputs or outcomes); and
--productivity (unit of benefit per full time employee or field office).
Program designers also should consider the program's social efficiency; i.e., how the program might be designed to achieve the greatest net benefit to society as a whole. In this context, benefits and costs are defined more inclusively than in analyzing cost-effectiveness, although there probably will be some overlap. To determine efficiency, analysts would employ the standard techniques of benefit-cost analysis to estimate the net social benefits of one or more program designs. Steps include:
--identifying major relevant impacts of the policy, including direct and indirect impacts on the program's direct beneficiaries, other affected groups, and society as a whole;
--categorizing costs and benefits for various affected groups; and
--quantifying dollar impacts.
In principle, the program design which is estimated to achieve the highest net benefit should be selected. However, optimizing distributional benefits (to either primary or secondary beneficiaries) typically conflicts with optimizing aggregate social welfare. Some times this may be appropriate (i.e., targeted programs). At other times some might criticize the selection of sub-optimally efficient programs as pork-barrel spending. Making explicit the trade-offs inherent in each design can help inform the debate.
5. Evaluate Consistency with Accepted Management
Principles
While the field of management is not a precise science, it does represent an accumulation of knowledge useful in illuminating the strengths and weaknesses of public programs. Using generally accepted management principles to evaluate various program designs may help identify both inherent deficiencies and alternative remedies. Where violations of conventional wisdom are appropriate, the justifications should be clear and convincing.
Among the numerous management principles applicable to program design to be given particular emphasis are those consistent with the core values of reinventing government, namely:
--serving customers,
--empowering employees,
--empowering communities to solve their own problems, and
--fostering excellence.
For example, program designers should consider the following reinvention principles that improve the efficiency and effectiveness of their programs:
--create a clear sense of mission,
--steer more, row less,
--delegate authority and responsibility,
--replace regulations with incentives,
--develop budgets based on outcomes,
--inject competition where appropriate,
--search for market, not administrative, solutions, and
--measure success by customer satisfaction.
6. Ensure Financial Feasibility
Given the increasingly strong drive to reduce the federal budget deficit, and competing claims on limited funding, affordability is an inevitable criterion for evaluating new and existing public programs. Although not all programs have direct budgetary impact, many do. Typically, cost calculations and estimates can be difficult to derive. Accounting methods interact with funding streams and auditing requirements in ways that are not always sensible or consistent.
Nevertheless, the program cost must be known if the government is to avoid incurring unlimited liabilities. A program's budgetary costs can be determined in a variety of ways, each of different interest to different constituencies. Among the alternate cost elements that may be relevant are:
--total life cycle costs: the full costs of creating, operating, maintaining, and terminating the program from inception to conclusion (where end points are reasonably defined);
--annual budget requirements: the amount that must be appropriated annually to sustain/maintain the program throughout its lifetime;
--risk of cost overruns: the probability that actual program costs will exceed approved budget levels including the expected cost of unbudgeted contingent liabilities;
--revenue generation: the expected income return to the Treasury from the program's operation; and
--debt management: the ratio of the funds collected or recaptured to the amount owed the U.S. Treasury.
7. Determine Feasibility of Implementation
Even the best designed program will not be effective in its execution if fatal flaws of implementation are not considered during its design. Unfortunately, those persons most familiar with implementation do not usually play key roles in design, depriving policy officials of the opportunity to obtain field validation and assessment before final decisions on design. Without effective implementation, program proponents will not see their goals successfully accomplished.Given the number and types of federal programs, it is unrealistic to formulate universal principles of implementation that apply exactly to all programs. Yet it may be instructive to identify a few common pitfalls to consider in the designs of many types of public service programs.
Foremost, implementation should be embedded in the program design.[Endnote 5] Therefore, the more that programs explicitly consider anticipated difficulties and conditions of implementation, the fewer the surprises and the more likely program success. Although detailed operational plans are not usually developed during the program design phase, at least some consideration in broad outline would be prudent management and politics. However, implementation must not compromise or obscure the program's basic purpose and integrity. If political realities so dilute the program's integrity that the original intent is no longer served, then managers should abort the design rather than succumb to the temptation of just getting anything approved.
Second, it is imperative to identify clearly who is ultimately responsible for the program to anticipate their capacity to administer and to ensure accountability. We cannot know the barriers and impediments to success without knowing clearly who (which organization(s)) is charged with carrying out the program. An objective assessment of the capabilities of the agencies through which the program resources and responsibilities will flow could pinpoint potential roadblocks, sources of conflict, and choke points that might hamper implementation. And, as trite as it sounds, those responsible for implementing a program must know what they are supposed to do. Policies, if not clear, accurate, and consistent, may cause misunderstanding and confusion, and confer discretion that may be roundly abused. For federal programs to be implemented more effectively, we must ensure the following:
(a) those bearing the major responsibility for
implementation are aware of what isactually happening
in another unit of government, a field agency, or the
private sector;
(b) those responsible for implementation earnestly
strive to achieve the stated program goals and
mission;
(c) policies and actions are decisively made and
clearly transmitted; and
(d) decisions and policies encourage creativity,
flexibility, and adaptability.
Third, an accurate description of the organizational network directly affected by the program would identify the critical relationships and points of coordination required or implied by program operations. This might include a flow chart of transactions depicting key relationships. Omissions and redundancies are more easily and economically remedied in the planning stages than later. Implementing policies and programs requires a coordination of efforts between various groups and organizations, particularly regarding complex policies or decisions.
Fourth, the timing of critical events is absolutely essential. Although many alternate sequences of activities are possible, key deadlines may not be discretionary (e.g., budget approvals, legislative requirements, awarding of contracts, etc.) and certain schedules offer distinct advantages over others. More programs suffer from overly ambitious timetables than from unduly conservative ones; effective implementation requires sufficient time.
Fifth, in designing programs, attention must be given to resources. However, due to various political compromises and conflicts, sufficient resources often are not available to implement a program properly. Program designers may not be aware of inadequate resources or may choose to ignore the problem. There are several important components of resources critical to implementation, including:
(a) adequate staff with the appropriate knowledge
(programmatic and managerial);
(b) sufficient information on how a program or
decision is to be implemented and the
support/approval of other necessary agencies and
participants;
(c) necessary authority to permit the programs or
decisions to be implemented; and
(d) facilities and equipment necessary to implement
the program or decision.
Sixth, prudent experimentation before committing the government to a major public investment could help avoid expensive misadventures and illuminate the pitfalls in uncharted domains. While pilot demonstrations are not appropriate for all programs, they are highly desirable under a wide range of conditions such as:
(a) high risk programs--where the consequences of
ailure are catastrophic or life threatening;
(b) where the economic investment is substantial
enough to justify small experiments;
(c) where there is significant complexity--either
organizational or programmatic (e.g., health care,
homelessness, economic renewal);
(d) where there is great uncertainty surrounding the
proper solutions;
(e) where there is intense public controversy or low
political consensus; and
(f) when there is no immediate urgency that precludes
more deliberate exploration of alternate approaches.
8. Provide for Program Flexibility
Perhaps the only universal element in all federal programs is the certainty that even the most carefully formulated plans and policies will not be realized exactly as designed. Given the inordinately long gestation periods for many public programs, coupled with lengthy implementation periods, it may be as long as five years before a program may be mature enough to sustain a valid evaluation.
In the meantime, the conditions and constraints under which a program operates may have changed significantly from when it was first conceptualized. Therefore, the design of programs should permit enough management flexibility to allow agencies to adapt to external changes, unforeseen circumstances, variations in resource levels, and schedule changes. In addition, program designs should free implementing organizations to tailor programs to local circumstances. Where initial rigidity in structure is necessary, waiver authority is essential to accommodate exceptional cases and unanticipated conditions.[Endnote 6]
While no one can anticipate every possible contingency, the most likely changes (e.g., budget erosion, schedule slippage, personnel shifts) should not have catastrophic impacts on well-designed programs. Programs dependent on critical, highly sophisticated, specialized, or scarce resources (e.g., key individuals, unproven technology, rare expertise, unique facilities) are fragile because they are vulnerable to uncertainties in the availability, quality, and quantity of these essential resources. For example, many attempts to modernize automated information systems often under- or over-estimate the availability of state of the art technology, resulting in either investments in obsolete hardware/software or costly delays waiting for technology to catch up.
Two especially critical flexibilities involve schedule and budget. With the enormous vagaries of when and what will survive the approval processes both in the executive and legislative branches, and the uncertainties of program execution, good program designs must withstand significant variations in time (i.e., date of initiation, rates of progress, completion milestones, etc.) and budget availability (both total amount and annual appropriations). Programs should be designed to absorb reasonable schedule changes with minimal impact. Similarly, robust programs should yield acceptable returns on investment over a wide range of program size (i.e., they should be fairly elastic over different scale levels). Programs that are designed as all or nothing versus those that can be bought by the yard are especially brittle in the face of inevitable uncertainties in the appropriations and authorizations processes.
9. Institute Performance Measurement and Program
Evaluation
Program designs should incorporate feedback mechanisms enabling managers and policymakers to assess how well a program works in terms of both implementation and achievement of program goals. Of course, program designers first must sufficiently identify the program's goals and specific performance objectives to measure outcomes.
Take, for example, the Customs Service's trade law violation detection program. In fiscal 1991, U.S. Customs did not detect about 84 percent of the estimated trade law violations in imported cargo.[Endnote 7] Does a detection rate of 16 percent indicate success? A strong design should require benchmarks for success/failure and delineate when a program should be redesigned, concluded because successful, or terminated for failure.
Designers, in consultation with potential users of the performance data, should monitor a range of key performance indicators.[Endnote 8] A wide range of types of data could be collected and reported, including a program's:
--inputs (e.g., funding and staff,
beneficiary characteristics),
--workload,
--outputs (services/final products),
--outcomes of products or services (i.e., for a job training program, the number ofjob placements that result in a year's employment),
--efficiency and/or productivity,
--beneficiary satisfaction,
--employee satisfaction, and
--service quality and timeliness.
A performance measurement system also requires identified data sources and reliable collection and reporting methods. The value of the information provided should be carefully weighed against the costs of data collection, including the burden imposed on reporting entities (such as state and local governments). Where practical, performance reports should be designed to meet the diverse information needs (in terms of both quantity and particular indicators presented) of the various users of performance data (e.g., beneficiaries, line staff, and program managers, senior agency officials, OMB, etc.). Nonetheless, compromises among users may be unavoidable.
The new Government Performance and Results Act of 1993 will institutionalize performance measurement throughout the federal government. Key elements of this legislation include strategic plans, performance plans, performance reports, managerial flexibility waivers, and performance budgeting. This Act requires a variety of pilot projects which could provide fruitful opportunities for the application of the several program design criteria suggested here.
10. Build in Cessation Provisions
There is a common perception that some government programs and organizations outlive their usefulness yet continue to exist without rational purpose or tangible benefit. Government has often been criticized as being totally incapable of abandoning programs, despite success, failure, or lack of evidence of either. Many government programs do not explicitly provide for their conclusion or termination for any cause whatsoever. A program design which does not address the ultimate fate of the program is at best incomplete, at worst selfperpetuating. Although some programs have no expected expiration (e.g., Social Security), others do possess an ultimate end point (e.g., inoculation programs should cease when the disease is essentially eliminated, as in the case of smallpox). Determination should be made at the onset as to what marks the success or failure of a program. When these benchmarks are established, it will be easier to decide when to redesign a program or just stop.
The Helium Fund Program was created in 1925 to ensure helium supplies for blimps. It has accumulated a 176- year supply of helium along with a $1.4 billion debt. All original objectives (except paying for itself) have been met, and private suppliers could meet all federal needs at lower cost.[Endnote 9] The obvious question is who are the primary beneficiaries of this program now that blimps are not critical to national security or commerce? Moreover, the helium program lacks any cessation provisions, an all too common failing in program design.
Sunset laws have a checkered history.[Endnote 10] While enjoying a surge of popularity at the state level in the 1970s and early 1980s, sunset laws have produced mixed results and many states are repealing ineffective sunset legislation. In January 1993, a sunset bill (S. 186) was reintroduced in the 103d Congress which would require formal reauthorization of federal programs every ten years. Given the amount of controversy around the applicability of sunset requirements to federal agencies, the bill's future is uncertain. Perhaps more promising avenues exist in strengthening reauthorization requirements by incorporating rigorous performance measurements and enforcing appropriate discipline in both the executive and legislative branches of government.
What Can be Done to Develop and Promulgate Program Design?
Program design criteria are merely the foundation for more efficient and effective federal programs. To establish a credible discipline of program design, the criteria and the technology used in successful design should be developed more fully throughout the federal government. The following are recommendations for developing a formal discipline of program design.
DES01: Activate Program Design as a Formal Discipline
(2)
The President should direct the President's Management Council (PMC) to sponsor as a high priority project the development and publication of a comprehensive handbook that will enable federal managers and policymakers to understand the strengths and weaknesses of alternative program designs. [Endnote 1] This policy research should include but not be limited to:
--a complete review of the literature relevant to program design (e.g., program evaluation, public policy, policy implementation, public administration);
--the development of a comprehensive taxonomy of existing federal programs to highlight the important structural differences and similarities across the government;
--the identification of the different design approaches (i.e., tools and modalities) used in federal programs to serve the public;
--an assessment of the comparative advantages of various program designs under different circumstances across several federal agencies;
--a compilation of analyses of representative case studies that illustrate the consequences of superior and inferior program designs; and
--the creation of model designs for each program type (e.g., grants, loans, insurance, regulatory mandates, tax preferences).
This initiative should be completed on an expedited basis with the first products to be delivered within one year and the final products delivered no later than three years. This project would be funded through the National Science Foundation. External participants in this effort should include established sources of relevant expertise such as the National Academy of Public Administration, the Brookings Institution, universities, and other public policy analysis organizations.
A governmentwide conference on program design could jump start the development of program design by convening agency representatives, congressional staffers, state and local constituencies, academics, and other policy experts. The conference would have three objectives:
--survey the extent and quality of existing program design knowledge;
--identify specific opportunities for education and research; and
--develop a blueprint for a decade-long, governmentwide training strategy.
DES02: Establish Pilot Program Design Capabilities in
One or Two Agencies (2)
After development of the handbook, the PMC should designate one or two agencies to pilot test program design capability to determine the value and costs of this function for a reasonable probationary period such as two years. For example, these pilots could establish a modest program design office in each agency staffed by personnel qualified in program design, management, and evaluation. The purpose of these offices would be to help senior officials design new programs, assess alternate program designs, and review existing programs for design integrity. Other mechanisms and structures for creating program design capability would be permissible. Where those pilots prove successful, program design capabilities would be expanded to other agencies on an incremental basis. However, if they fail to meet predetermined performance criteria, the pilots would be restructured or terminated.
The long-term ideal for institutionalizing sound program design is the integration of good design practices in the program formulation and management processes organic to each agency and department. The most significant advantage to establishing program design functions at the agency level is that it brings program design capabilities closer to the customers and program managers. If reinventing government is to be successful, there must be a comprehensive cultural change in the agencies. The placement of the program design function at the agency level should help to facilitate cultural change where many program designs originate and where programs are implemented.
A principal disadvantage of this approach is that the design staffs may be unduly constrained and influenced by their immediate environments. They may find it difficult to elevate their perspectives to a higher plane that would foster fresh, creative approaches to the design of agency programs. Also, innovative inter-agency opportunities may be harder to conceive and implement if the program design function is totally embedded within a single agency. However, the pilot demonstrations should indicate if this is an insurmountable handicap or if feasible remedies are available.
All agencies should be encouraged to test program design principles in their reinvention laboratories. Principles could be applied to either an existing program or to the design of a new program. The reinvention laboratory would provide a protected setting in which to examine the possibility of improved outcomes based on improved design features. Some agencies (e.g., Interior) have emulated the National Performance Review structure and have program design teams. The agency team can provide advice and assistance to the laboratory regarding the proper use of design principles and the evaluation of success or failure.
DES03: Encourage the Strengthening of Program Design
in the Legislative Branch (2)
To enhance congressional program design performance, the PMC should:
--work with the Congressional Research Service (CRS) to include consideration of program design concepts in all relevant training and materials provided by CRS; and
--encourage the participation of CRS, key congressional staff, the Congressional Budget Office (CBO), and the General Accounting Office (GAO) in program design courses that are developed under the next action (DES04).
Many, if not most, federal programs originate in Congress. Yet program design as a concept or a function is given scant attention there. Among the offices that participate in the preparation of legislation, varying degrees of emphasis are placed on program design. And there is no general, consistent pattern of the use of these offices. The principal drafters of legislation--the Offices of the Legislative Counsel of the Senate and the House-- while they do not make program policy, generally do provide a legal framework and historical context that provide a basis for the program design process. CRS, GAO, and CBO often play significant roles in providing analyses and information used to prepare legislation and in reviewing and shaping draft legislation, and thus contribute to program design.
CBO's role is generally limited to analysis of the fiscal implications of proposed legislation. Evaluation of cost-effectiveness and efficiency of a proposed program as well as its financial viability might be an appropriate expansion of the traditional CBO role. CRS also offers extensive training programs and provides research materials to congressional staff on a wide range of subjects, many of which could have significant program design implications. If all these offices were able and willing to provide expert advice on program design, members and staff of Congress would be better equipped to apply the recommended program design criteria.
Strengthening program design capabilities and encouraging greater use of available resources in the legislative branch would not only improve program design, it would also enhance the formulation of programs by framing the debates which occur on and off the floors of both Houses. Ideally, the individuals developing program design in Congress would cooperate on the design of new programs as well as share expertise concerning tools of public action and obstacles to solid program design. This cooperation would also be extended by congressional offices and staff to the executive branch at the agency level and to the Executive Office of the President and could perhaps reduce some of the traditional friction between these two branches of government.[Endnote 2]
DES04: Commission Program Design Courses (2)
In conjunction with its sponsorship of research and development on program design, the President's Management Council should commission the development of a family of program design courses to educate and train several generations of current and future policymakers, program designers, and managers. If informed, reasoned program design is to flourish throughout the federal government, training and education will be important factors in that evolution. Given the wide variation in design knowledge and needs across government, a multitiered approach to education would be superior to a single course of instruction for everyone. A comprehensive education strategy for program design might include:
--a series of short seminars (1-3 days), each focused on a particular mode of public service delivery such as federal subsidies, tax incentives, voucher systems, government sponsored enterprises/corporations, privatization of services, loan guarantees, regulatory initiatives, etc. The target audience for such courses would be agency program specialists (GS 12-15), congressional staffers, budget examiners, and auditors who have need for in-depth knowledge of a specific type of program design and its strengths and weaknesses. These seminars could be disseminated through agency training programs, the Office of Personnel Management's (OPM) executive development centers, the Department of Agriculture Graduate School, and commercial/non-profit institutions such as the Brookings Institute, National Academy of Public Administration, the Congressional Management Foundation, etc.;
--a longer residential course (5-10 days) which would provide a comprehensive overview of program design covering several different design modes comparing the relative advantages and disadvantages of each. The target audience for this course would be program generalists or senior government policymakers who require broader exposure across a variety of programs. This course could be available through the Federal Executive Institute, the Federal Quality Institute, nonprofit organizations, and/or academic institutions;
--introduction of program design subjects into existing management development programs such as agency executive development courses, OPM seminars, senior military schools, and Senior Executive Service candidate development programs; and
--a variety of academic offerings at graduate schools of public administration, public policy, government, and political science. To reach future generations of public servants, the design of government programs should be taught in graduate curricula. If visible academic institutions were to include program design courses as academic electives or requirements, future civil servants would be better educated and vigorous research may begin to emerge as faculty and graduate students collect, synthesize, and create material for their courses.
Schools of public policy and administration should heed the advice of Professor Herbert Simon, Nobel Laureate in Economics, who suggested a more active role for universities in teaching design:
The professional schools will reassume their professional responsibilities just to the degree that they can discover a science of design, a body of intellectually tough, analytic, partly formalizable, partly empirical, teachable doctrine about the design process . . . [S]uch a science of design [is] not only possible but is actually emerging at the present time. It has already begun to penetrate the engineering schools, particularly through programs in computer science and "systems engineering," and business schools through management science. [Endnote 3]
Appendix A:
Summary of Actions by Implementation Category
(2) President, Executive Office of the President, or
Office of Management and Budget can do
DES01: Activate Program Design as a Formal
Discipline
DES02: Establish Pilot Program Design Capabilities
in One or Two Agencies
DES03: Encourage the Strengthening of Program
Design in the Legislative Branch
DES04: Commission Program Design Courses
Accompanying Reports of the National Performance Review
Governmental Systems..........................Abbr.
Changing Internal Culture
Creating Quality Leadership and Management....QUAL
Streamlining Management Control................SMC Transforming Organizational Structures.........ORG Improving Customer Service.....................ICS
Reinventing Processes and Systems
Mission-Driven, Results-Oriented Budgeting.....BGT Improving Financial Management..................FM Reinventing Human Resource Management..........HRM Reinventing Federal Procurement...............PROC Reinventing Support Services...................SUP Reengineering Through Information Technology....IT Rethinking Program Design......................DES
Restructuring the Federal Role
Strengthening the Partnership in
Intergovernmental Service Delivery.............FSL Reinventing Environmental Management...........ENV Improving Regulatory Systems...................REG
Agencies and Departments
Agency for International Development...........AID Department of Agriculture.....................USDA Department of Commerce.........................DOC Department of Defense..........................DOD Department of Education.........................ED Department of Energy...........................DOE Environmental Protection Agency................EPA Executive Office of the President..............EOP Federal Emergency Management Agency...........FEMA General Services Administration................GSA Department of Health and Human Services........HHS Department of Housing and Urban Development....HUD Intelligence Community.......................INTEL Department of the Interior.....................DOI Department of Justice..........................DOJ Department of Labor............................DOL
National Aeronautics and Space Administration.NASA National Science Foundation/Office of Science
and Technology Policy..........................NSF Office of Personnel Management.................OPM Small Business Administration..................SBA Department of State/ U.S. Information Agency...DOS Department of Transportation...................DOT Department of the Treasury/ Resolution Trust Corporation..............................TRE Department of Veterans Affairs.................DVA