U.S. Department of State

Strategic Goal 1

Regional Stability

Title or Topic

Security Assistance for New NATO and NATO Aspirant Countries

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

EUR - Nerissa Cook

Major Findings

• Program was given a PART rating of "Moderately Effective.".
• Program purpose is clear.
• State and Defense Departments formally review strategic goals in a highly structured processes.
• Independent parties do not carry out regularly scheduled evaluation of program effectiveness. The Department and DoD may have differing priorities for the programs, and the two departments' budget schedules do not produce simultaneous recommendations.
• Program results in many of the countries are positive.

Major Recommendations

• State and Defense Departments will press nations that are lagging in their reform efforts.
• Continued development of an e-government management tool will assist management in seeing where deficiencies appear.

Major Actions Taken or That Will be Taken

• Specific long-term goals and short-term goals and performance indicators identified via PART will be integrated into the FY 2005 Bureau Performance Plans.

Title or Topic

Security Assistance Programs for Sub-Saharan Africa

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

AF/RA - Michael J. Bittrick

Major Findings

• The program was given a PART rating of "Results Not Demonstrated."
• Performance plans set out long-term goals with associated annual targets, but no annual goals.
• Performance goals are overly broad, making it difficult to tie the outcome to the specific program.
• Program manager reports and program evaluations should be better linked to specific goals in performance plans.
• Actual performance information should be included in annual performance reports to compare annual and long-term goals to actual performance.
• Performance measurement contracts or other means of manager accountability should be implemented.

Major Recommendations

• Decrease funding due to difficulty measuring program effectiveness.
• Restructure performance plans to include separate annual and long-term goals, targets and baseline information. Standardize performance information collection.
• Implement measures to hold program managers accountable for program performance.

Major Actions Taken or That Will be Taken

• Specific long-term goals and short-term goals and performance indicators identified via PART will be integrated into the FY 2005 Bureau Performance Plans.

Title or Topic

Peacekeeping Operations (including OSCE and East Timor Programs)

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

EAP - Richard DeVilla Franca

Major Findings

• The program was given a PART rating of "Results Not Demonstrated."
• Performance plans set out long-term goals with associated annual targets, but no annual goals.
• Performance goals are overly broad and dependent on a number of factors, making it difficult to tie the outcome to the specific program.
• Program manager evaluations should be better linked to performance goals and performance plans.
• Clarify which program manager(s) is/are ultimately accountable for program performance.

Major Recommendations

• Decrease funding due to difficulty measuring program performance.
• Restructure performance plans to include separate annual and long-term goals, targets and baseline information. Performance information should be collected in a standardized manner

Major Actions Taken or That Will be Taken

• Decrease funding due to difficulty measuring program performance.
• Specific long-term goals and short-term goals and performance indicators identified via PART will be integrated into the FY 2005 Bureau Performance Plans.

Strategic Goal 2

Weapons of Mass Destruction

Title or Topic

Nuclear Nonproliferation: U.S. Efforts to Help Other Countries Combat Nuclear Smuggling Need Strengthened Coordination and Planing

Evaluator

General Accounting Office

Department Contact

NP - Michael Foughty

Major Findings

• Current multi-agency approach is not well coordinated. No agency leads efforts to prioritize funding and assess recipients' requirements.
• Within agencies, including the Department, there are also coordination problems.
• While equipment transfers have improved recipients' ability to detect nuclear smuggling, some equipment is not well maintained, accounted for or installed on a timely basis.
• There is no systematic approach to obtaining information on smuggling incidents in recipient countries.

Major Recommendations

• Department should take the lead in developing a government-wide plan to help other countries develop an integrated approach that prevents nuclear smuggling. The plan should include unified USG program goals; definition of agencies' roles; program-wide cost estimates, timeframes, performance measures, equipment maintenance strategies and exit strategies for each country.
• Agencies should consolidate their efforts internally: the Department should establish one program office.
• Strengthen efforts to ensure accounting, maintenance and timely installation of equipment, and ensure that information about nuclear materials detected by US-provided equipment is shared with the U.S.

Major Actions Taken or That Will be Taken

• Department is leading interagency development of a strategic plan for nuclear nonproliferation assistance within a new International Nuclear Detection Interagency Working Group, established in July 2002, that will meet quarterly.
• Strategic plan for nuclear detection assistance has been prepared, including all of the elements recommended by GAO and more. As part of the process, DOE is preparing a global threat analysis to better inform planning.
• All Department radiation detection assistance has been consolidated in one office.
• DOE has agreed to accept Department funding to undertake maintenance, repair and replacement of detection equipment provided by all agencies for non-proliferation purposes.

Title or Topic

Chemical Weapons: Organization for the Prohibition of Chemical Weapons Needs Comprehensive Plan to Correct Budgeting Weaknesses

Evaluator

General Accounting Office

Department Contact

AC - Jamie Young

Major Findings

• Since establishment in 1997, the ability of the OPCW to carry out key inspection functions has been hindered by inaccurate budget projections and, more recently, budget deficits.
• The OPCW has underestimated expenses and overestimated real income from member states.
• This has led to significant reductions in inspection activity (primarily at industrial, as opposed to military sites) in 2001 and 2002.
• Although the OPCW and the Department have taken some steps, the OPCW has not developed a comprehensive plan to overcome its inherent budgeting weaknesses.

Major Recommendations

• Department should work with the OPCW to develop a comprehensive plan to address budgeting weaknesses.
• Department should report annually to Congress on progress in correcting these budgeting weaknesses.

Major Actions Taken or That Will be Taken

• Numerous measures were taken by the Department at the October 2002 Conference of States Parties that will:
• Restore normal levels of inspection activity for the last quarter of the calendar year.
• Improve the accuracy of income and expenditure projections for 2003.
• Provide OPCW with greater liquidity to cope with temporary cash-flow problems.
• The Department will work with the new Director General of the OPCW to develop a comprehensive plan that assesses the efficacy and adequacy of measures already taken, continues improvement of OPCW budgeting, and institutionalizes changes that are shown to be effective.

Strategic Goal 7

American Citizens

Title or Topic

Office of American Citizen Services and Crisis Management (ISP-I-02-07 2/02)

Evaluator

State Department Inspector General

Department Contact

CA - Leo Wollemborg

Major Findings

• The Office of American Citizen Services and Crisis Management (ACS) rightfully enjoys an excellent reputation for providing the traveling public with a high level of service and timely, accurate information and giving consular sections support and guidance.
• Additional attention to management issues, office space, automation, and staff development would improve operations.

Major Recommendations

• The OIG recommended improvements in standard operating procedures, written guidance, training and employee development, management, and information technology tools.

Major Actions Taken or That Will be Taken

• The recommendations have all been accepted and implemented by CA. ACS updated and improved its orientation and training for new officers, written reference materials (including the Foreign Affairs Manual), and its computer programs. ACS also improved management procedures to more effectively utilize the skills and talents of senior Civil Service employees.

Title or Topic

Review of Domestic Passport Operations (ISP-CA-02-32 3/02)

Evaluator

State Department Inspector General

Department Contact

CA - Leo Wollemborg

Major Findings

• Although Passport Services has established a comprehensive set of internal control procedures for cashiering and blank passport book controls that minimize the risk of cashiering and blank passport book malfeasance, the internal control procedures for cashiering were not implemented consistently at five of the six passport agencies visited.
• CA's management assessment/internal control reviews are not conducted frequently enough to assess adequately whether internal controls have been properly implemented.

Major Recommendations

• OIG recommended that Passport Services review and make changes to internal controls procedures at passport agencies to improve the tracking and monitoring of cash receipts and deposits, and that management assessment/internal controls reviews occur at each passport agency every other year.

Major Actions Taken or That Will be Taken

• Passport Services developed an internal controls compliance program to ensure that agencies promptly comply with report recommendations.
• Passport Services increased internal controls reviews of passport agencies to once every two years, as recommended.

Title or Topic

Information Management: Selected Agencies' Handling of Personal Information GAO-02-1058 September 30, 2002

Evaluator

General Accounting Office

Department Contact

CA - Colombia Barrosse

Major Findings

• The agencies generally complied with the key requirements and guidance pertaining to information collection, privacy, security, and records management. However, GAO identified isolated instances of forms that were not accurate or current or did not contain the proper privacy notices. Specifically, at the Departments of Labor and State, the forms displayed on the Internet showed expired OMB approvals.

Major Recommendations

• None for Department

Major Actions Taken or That Will be Taken

• The DS-11 passport form displayed on the Internet showed an expired OMB approval. The renewal paperwork was approved by OMB after the GAO's initial meetings with Department. An appropriately updated DS-11 is now on the Internet.

Strategic Goal 8

Travel & Migration

Title or Topic

Consular Affairs Information Security Program (IT-A-02-04 9/02)

Evaluator

State Department Inspector General

Department Contact

CA - Leo Wollemborg

Major Findings

• CA has developed a comprehensive approach to addressing information security risks. CA's 1999 Modernized Systems Information Systems Security Plan establishes organizational authorities and responsibilities to ensure that specified security requirements are met in its client-server environment, domestically and overseas. CA oversees the development and dissemination of policies and procedures, ensures the development and presentation of user and contractor awareness sessions, conducts both vulnerability and risk assessments, and inspects and spot checks systems and desktops to confirm that consular sections are in compliance with required security configurations.
• Seventy percent of CA systems had risk assessments, almost fifty percent were operating with a security level determination and an overall security plan in place, and eleven percent had been certified and accredited.

Major Recommendations

• More needs to be done to ensure CA managers have sufficient information concerning the extent to which their systems are protected against fraud, illegal practices, or mission failure.

Major Actions Taken or That Will be Taken

• CA has completed categorizing and prioritizing applications and is actively working toward bringing each active system fully in line with applicable statutes and guidelines. Several more systems successfully completed certification; accreditation is pending. CA has a program for independent security review of its systems, and reviews of select overseas applications have been completed. Security requirements are being addressed earlier in an application's development process resulting in security requirements becoming verifiable and traceable throughout the system's life cycle.

Title or Topic

State Department Visa and Consular Services Program (Border Security)

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

CA - Betty Ng

Major Findings

• The program was given a PART rating of "Moderately Effective."
• While there are frequent evaluations of parts of the program and annual targets are regularly met, annual goals and targets do not adequately link to the long-term goals or provide relevant performance data. Measurable goals should better reflect what Department is actually doing to meet the needs of Americans traveling abroad and non-U.S. citizens traveling to the United States.
• Department is collecting adequate information to successfully manage and implement the program.
• Department has revised its planning in accordance with new laws and significant new requirements to integrate more effectively its planning with that of other agencies related to homeland security and counterterrorism.
• Program is not redundant and works in conjunction with many other federal programs and agencies.

Major Recommendations

• No funding recommendations since programs are funded primarily through consular fees through Department's Bureau of Consular Affairs, which is relatively effective and heavily scrutinized.

Major Actions Taken or That Will be Taken

• Specific long-term goals and short-term goals and performance indicators identified via PART will be integrated into the FY 2005 Bureau Performance Plans.

Strategic Goal 9

Countering Terrorism

Title or Topic

Anti-Terrorism Assistance Program

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

S/CT - Niki Grandrimo

Major Findings

• Program was given a PART rating of "Moderately Effective."
• Events have compelled the Department to improve strategic planning to make program responsive to on-site training delivery needs overseas.
• Course offerings have been expanded to cover new training needs including terrorist financing.
• Program collects performance information and allocates resources to specific activities to ensure utilization for intended purpose.
• Fiscal planning is constantly reviewed in context of new priorities outlined by the Department's Coordinator for Counter-Terrorism.
• Continued improvement in attempting to quantifiably measure the success of training programs is necessary to set targets and measure progress against long-term goals.

Major Recommendations

• Slightly increased funding to allow program to continue all training programs currently underway, but no significant expansion in the number of courses and eligible countries.
• Improve long-term performance measurement and establish measures to gauge progress toward long-term goals.

Major Actions Taken or That Will be Taken

• Specific long-term goals and short-term goals and performance indicators identified via PART will be integrated into the FY 2005 Bureau Performance Plans.

Strategic Goal 11

Illegal Drugs

Title or Topic

Evaluation of Model Programs for Drug Abusing High Risk Youth in Brazil and Peru

Evaluator

Outside Contractors

Department Contact

INL - John Bargeron

Major Findings

• Between fifty and seventy percent (Brazil) and fifty percent (Peru) of participants completed the program and returned to families or found jobs, as compared to a completion rate of only ten to fifteen percent for similar programs in the U.S.
• Cost for six-month residential treatment was $1,533 (Brazil) and $1,768 (Peru) as compared to $19,000 in U.S.

Major Recommendations

• Funding for programs should continue.
• Peru program needs a separate facility to separate adolescents in various stages of the program
• Client-staff ratio in Peru needs improvement.

Major Actions Taken or That Will be Taken

• INL will continue to fund the Brazil and Peru programs
• Peru program is in the process of adjusting the client-staff ratio
• Construction of a separate facility for Peru program will start when funding is available.

Title or Topic

Drug Abuse Treatment Training in Peru

Evaluator

Outside Contractors

Department Contact

INL - John Bargeron

Major Findings

• Staff increased its use of actual tools and principles from the training.
• Training methods were faithfully implemented at one- and nine-month follow-ups after the training.

Major Recommendations

• Continue providing the drug abuse treatment training to organizations in foreign countries that offer substance abuse treatment services.
• Prepare a student training guide or manual that highlights key concepts and terms, and connects the content to the training objectives.
• Conduct a future evaluation by following-up on clients of the three groups to ascertain if there is any group difference in former clients' quality of life and drug use relapse as measures of organizational effectiveness.

Major Actions Taken or That Will be Taken

• INL continues to provide the drug treatment training to foreign countries.
• The content of the student training manuals has been revised as recommended by the evaluator.
• INL is now funding a long-term, follow-up evaluation of the training program's effectiveness on impacting relapse, crime and reintegration rates among clients.

Strategic Goal 13

Humanitarian Assistance

Title or Topic

Humanitarian Response Efforts for Women and Conflict Victims

Evaluator

State Department Inspector General

Department Contact

PRM - Kelly Clements

Major Findings

• Progress made in advancing humanitarian response goals for refugee, returnee, and internally displaced women.
• Performance monitoring needs to support program and funding decisions more frequently.
• Improved coordination with Department's implementing partners is necessary.

Major Recommendations

• PRM should establish performance objectives and indicators for all USG-funded IO and NGO activities addressing women's policy and program goals, including those for sexual violence and psychological social distress. Monitoring, evaluation, and funding of proposals should be based on these performance objectives and indicators. An automated database for retrieving and analyzing proposals, projects and progress toward meeting policy and program objectives should be on line by the end of FY 2002.
• PRM should coordinate with UNHCR to develop comprehensive guidelines for women's initiatives, including a charter that addresses its empowerment objectives, integration with other post-conflict activities, and the framework for transition to local women. PRM should take into account lessons from previous studies of women's initiatives.

Major Actions Taken or That Will be Taken

• PRM and UNHCR both support the Reproductive Health for Refugees Consortium (RHRC), a group of seven organizations established to increase access to a broad range of quality, voluntary reproductive health care to refugees and IDPs. The RHRC's draft multi-sectoral assessment tool, which is now being field tested, will include information on how best to monitor and evaluate projects and provide a guide for which objectives and indicators are best under certain circumstances. PRM has begun planning with the Department's Information Resources Management Bureau to implement the database.
• UNHCR is revising its Guidelines on Prevention and Response to Sexual Violence Against Refugees. PRM has provided comments to the shared drafts in order to strengthen these guidelines. PRM supported an assessment by the Women's Commission- for Refugee Women & Children of UNHCR's implementation of its Guidelines on the Protection of Refugee Women.

Title or Topic

UNHCR Policy on Refugee Women and Guidelines on Their Protection: An Assessment of Ten Years of Implementation

Evaluator

Women's Commission on Refugee Women and Children

Department Contact

PRM - Kelly Clements

Major Findings

• Implementation of the Guidelines on the Protection of Refugee Women was found to be uneven, incomplete, and occurring on an ad hoc basis in certain sites rather than in a globally consistent and systematic way.
• Past evaluations have recommended that training on gender be mandatory for all UNHCR staff, but the agency has chosen not to require this. Inadequate training impedes implementation of the Guidelines.

Major Recommendations

• Restore funding, which has now fallen below adequate levels, to ensure progress in implementation of the Policy and Guidelines on Protection of Refugee Women.
• Make mandatory training for all UNHCR staff on Guidelines a priority.
• In anticipating the impact of funding decisions, pay more attention to the linkages between assistance and protection for refugee women.
• Continue to advocate and promote policies and efforts that ensure protection for refugee women and improved gender programming within UNHCR.

Major Actions Taken or That Will be Taken

• In the FY 2004 Bureau Performance Plan, one PRM strategy is to target funding to projects that promote women's equal access to resources, and their participation in managing these resources, with particular focus on the distribution of food and other support items.
• PRM has funded UNHCR projects in Africa that focused on women's direct and indirect participation in managing and distributing resources, and that provided training to women in refugee law to help them better understand their rights as refugees.
• PRM funds an NGO partner to provide SGBV technical assistance to international UN agencies, NGOs, and host governments in a variety of locations with a goal of improving SGBV prevention and response among war affected populations.
• PRM will require UNHCR and other partners to adopt and implement codes of conduct that prohibit exploitation.

Title or Topic

Independent Evaluation of the Kosovo Women's Initiative

Evaluator

Consultants Jock M. Baker and Hilda Haug under the supervision of the UNHCR Evaluation and Policy Analysis Unit

Department Contact

PRM - Kelly Clements

Major Findings

• PRM's pro-activeness in supporting Kosovo Women's Initiative (KWI) is a positive factor.
• PRM's emphasis on rapid disbursement of fund and bi-monthly reports was unrealistic, inconsistent with KWI goals, and undermined the project's cost-effectiveness.
• The scale of resources that the USG earmarked for KWI ($10m) was out of proportion to UNHCR's capacity to implement the project.
• The multiple layers of bureaucracy and reporting complicated and confused project implementation.
• The large size of the earmark led UNHCR to question whether the donor was overly active in setting UNHCR's own priorities.

Major Recommendations

• The report recommends several possible alternate arrangements for future women's initiatives, including UNHCR as the lead agency, another UN agency as the lead, a lead bilateral agency, direct assistance to local NGOs, and implementation directly through an international NGO.

Major Actions Taken or That Will be Taken

• Department/PRM has not earmarked funding for the KWI in FY 2003. PRM will bear in mind the evaluation's recommendations when funding future women's initiatives. In particular, the critiques of onerous reporting requirements and the disproportional size of the earmark vis-a-vis the capacities of implementing partners should be considered in the planning for any future women's initiatives. Department/PRM is also incorporating the report's findings and recommendations into an ongoing lessons-learned project.

Title or Topic

Refugee Admissions to the U.S.

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

PRM - Kelly Clements

Major Findings

• Program was given a PART rating of "Adequate."
• Program scored well for planning; management and program managers collaborated with grantees and states to ensure effective use of funds.
• Strategic planning should narrow goals to measure program effectiveness more easily; draft FY 2004 Performance Plan includes more focused and quantifiable goals.
• Program has recently developed new performance measurements that do not yet have data available to measure progress in meeting targets; Administration expects performance in this area to increase in future years as managers begin managing by new measurements and as new performance data are available.
• There is some overlap between one function of the program and another program run by the Department of Health and Human Services.

Major Recommendations

• Budget increase to meet increased security costs.
• Administration will review relationship between programs at State Department and Department of Health and Human Services.
• Department will continue ongoing efforts to improve strategic planning to ensure goals are measurable and mission-related.

Major Actions Taken or That Will be Taken

• The FY 2005 Bureau Performance Plan will contain the specific long- and short-term goals and performance indicators identified via PART.

Title or Topic

Refugees to Israel

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

PRM - Kelly Clements

Major Findings

• Program was given a PART rating of "Adequate."
• Purpose of program is clear, but there is a declining need for the program as shown in number of individuals assisted yearly.
• The program has made significant progress in its strategic planning during the past year.
• The Department has worked with recipient organization on performance goals that meaningfully reflect program purpose.
• Overall, program is well managed. Department responded well to previous program evaluations.
• Program is making progress toward achieving its long- and short-term goals.

Major Recommendations

• Budget should reflect program's decreasing need.
• Annual goals for each of the program areas would be useful.

Major Actions Taken or That Will be Taken

• The FY 2005 Bureau Performance Plan will contain the specific long- and short-term goals and performance indicators identified via PART.

Strategic Goal 17

Mutual Understanding

Title or Topic

Outcome Assessment of the U.S. Fulbright Scholar Program

Evaluator

SRI International

Department Contact

ECA - Ted Kniker

Major Findings

• The evaluators found strong quantitative and qualitative evidence that the program is achieving its legislative mandate.
• Scholars build knowledge and long-term relationships with host institutions and foreign colleagues. Seventy-five percent of participants still collaborate with people met on their program, even fifteen to twenty years later.
• Scholars make U.S. campuses and communities more international. (73 percent have incorporated international aspects of their Fulbright experience into their courses and teaching).
• Scholars express high levels of satisfaction with the Fulbright Program.

Major Recommendations

• Provide more information about opportunities for alumni grants.
• Provide opportunities to participate in Fulbright panels.
• Give more attention to technical/scientific contributions made by Fulbright Scholars.
• Use retired alumni at low cost to deliver lectures, and other activities in their host countries.
• Organize travel/study trips for alumni.

Major Actions Taken or That Will be Taken

• Strengthened outreach to increase the diversity of participants within the Fulbright U.S. Scholar Program.
• More alumni activities and Involvement of alumni in follow-on program activities.
• Enhance outreach to alumni via the Fulbright Scholar Program website and Fulbrightweb.org (the online Fulbright community).
• Increase communication and coordination with the Fulbright Program Alumni Organization and its chapters.

Title or Topic

Outcome Assessment of the American Council on Young Political Leaders (ACYPL) Program

Evaluator

American Institutes for Research

Department Contact

ECA - Ted Kniker

Major Findings

• ACYPL Program is successful in giving U.S. and foreign participants a more complex understanding of international issues and their host country's political system, economy, culture, and social issues.
• The program is effective in creating institutional linkages and professional linkages among program delegation members and between delegates and their host country counterparts.
• The program promotes the Department's foreign policy objectives and strategic goals by encouraging mutual understanding between Americans and foreigners.

Major Recommendations

• Better prepare delegates about the host country prior to arrival.
• Increase the economic, cultural, and social components of the program.
• Allow delegates more free time to explore the host country.
• Develop an Internet-based system for alumni tracking.
• Increase ways that alumni can get involved in the program.
• Encourage the establishment of ACYPL counterpart organizations abroad.
• Ensure close collaboration of ACYPL counterpart organizations abroad with U.S. embassy staff.

Major Actions Taken or That Will be Taken

• Provide delegates with detailed briefing materials on program and host countries, orientations and meetings with embassy counterparts on logistical and administrative matters.
• ACYPL has stepped up efforts to ensure that economic, cultural, and social components are significant in each country visit.
• Study tours are rigorous but larger blocks of time are being added to include cultural activities, sightseeing and freetime.
• ACYPL is refining an alumni database and working with counterpart organizations to help track highly mobile alumni despite limited organizational resources.
• ACYPL will continue publishing the alumni newsletter and directory, hosting of new delegates, and begin new initiatives to further engage alumni.
• Old counterpart relationships have been re-established in two countries and new partnerships are being explored in as many as five new countries.
• ACYPL recognizes and encourages the need for good communication between its foreign counterpart organizations and U.S. embassy staff involved with the ACYPL program. ACYPL will continue to encourage and promote closer cooperation.

Title or Topic

Evaluation of the International Visitor Program Special Initiative Projects

Evaluator

ECA Office of Policy and Evaluation Staff

Department Contact

ECA - Ted Kniker

Major Findings

• The International Visitor Program Special Initiative projects were successful in increasing participants' understanding of the impact of the events of 9/11 on the United States and Americans and on global peace and security.
• The projects significantly contributed to the ECA and public diplomacy goals of promoting closer international relations and mutual understanding and respect.
• The program experience helped increase participants' understanding of the United States and Americans and of their professional fields and interests.
• The projects were successful in providing accurate and balanced information about the United States by providing opportunities for participants to dialogue with their U.S. professional counterparts as well as with "everyday Americans."

Major Recommendations

• At the start of their program, ensure that participants receive a thorough briefing on the U.S. government and the federal system.
• Build into the program more opportunities for participants to experience U.S. culture and society outside of formal meetings or presentations.
• Increase opportunities for participants to speak with students at schools, colleges, and universities as well as with civic groups in communities around the country.
• Secure more meetings with administration officials, particularly Department officials.

Major Actions Taken or That Will be Taken

• The program office awarded Meridian International Center a grant in FY 2003 to organize weekly presentations on federalism for visitors.
• The program office has instituted quarterly meetings with its eight-grantee national program ato discuss programming, pacing and other subjects. Through the National Council for International Visitors (NCIV) core grant, they will work with the Councils for International Visitors (CIVs) and the national program agencies (NPAs) on these issues in two regional NCIV meetings, the annual meeting in Washington, the programmers workshop in December 2002, and other training opportunities.
• Program office made an effort to publicize International Education Week among the CIVs to encourage them to use IVs in schools in their communities. Message is being reiterated to CIVs, NPAs and bureau staff in ongoing training sessions.
• Through branch chiefs and continued close coordination with public diplomacy office colleagues in the geographic and functional bureaus at Department, program office has been able to arrange more and higher level administration appointments for visitors. Afghan women's group and Arab women political leaders group projects were great successes in this regard.

Title or Topic

Muskie/FSA Graduate Program Evaluation

Evaluator

Iowa Social Science Institute, University of Iowa

Department Contact

ECA - Ted Kniker

Major Findings

The Muskie/FSA Program is meeting its goal of providing participants with the skills and experiences to become leaders in key sectors. Alumni are more likely than semifinalists to work in sectors of the economy targeted for reform - especially business and law. They have significant influence within the organizations where they work; they report significantly higher incomes, indicate greater job satisfaction, supervise significantly more employees, and evince more democratic ideas about workplace leadership than do semifinalists. They directly utilize skills and knowledge gained during their program in their current jobs. They report increased self-confidence, increased ability to overcome difficulties, and increased communication skills as a result of the program. Perhaps most importantly, they believe that they personally have a great deal of influence on society.

Major Recommendations

• The Program
• Interviewers from universities need additional training in ECA objectives and in using standardized interviewing protocols.
• More attention should be directed to civic education component of the program - i.e. getting participants more actively engaged in community activities during the program.
• The Post-Program Experience.
• Energy and resources need to be dedicated to keeping in touch with alumni.
• Alumni activities and organizations need to better address alumni needs and assist them in fulfilling their potential.

Major Actions Taken or That Will be Taken

• Interview protocols have been standardized; interviewers now receive training on FSA and ECA goals.
• Resources have been allocated to the administering organization to develop a better alumni-tracking system and to institutionalize the alumni contact strategy.
• Alumni programs now receive greater visibility, and more attention is given to soliciting alumni input in programming.

Title or Topic

Georgian Institute of Public Administration

Evaluator

Lehmann Surveys and Research

Department Contact

ECA - Ted Kniker

Major Findings

• GIPA is well run by the Georgian staff, effective, and exceeds its initial goals.
• The role played by the National Academy of Public Administration needs attention: better record-keeping, recruitment of a more diverse group of American faculty, and the creation of basic orientation materials.
• The institute has developed a reputation for excellence that is luring Georgia's best and brightest into the field of public administration. Alumni, a substantial proportion of whom choose employment in the public sector, are highly sought after in both the public and private sectors.

Major Recommendations

• Funding to support GIPA should be continued.
• Pay more attention to the American-side management by National Academy of Public Administration (NAPA).
• Additional funds should be allocated to support a case study workshop where alumni could learn how to turn their employment experiences into teaching materials for future cohorts, a small grants program to fund working papers in targeted issues of public policy and a small grant to fund the creation of a Georgian language textbook of public administration.

Major Actions Taken or That Will be Taken

• The recommendations will serve as a guideline for the solicitation to NAPA for another year of GIPA funding in 2003-2004.

Title or Topic

Assessment of Selected Educational Exchange Programs in Turkey

Evaluator

K.W. Tunnell Company and PDS, Inc.

Department Contact

ECA - Ted Kniker

Major Findings

• Participants learn much about the United States through their experience in the country. When they return to Turkey they understand the United States better and they continue to interact with U.S. institutions and individuals for many years into the future.
• Participants change as a result of the experience.
• Participants value the technical and educational benefits of their experience.

Major Recommendations

• Assess the alumni efforts for selected programs with a view toward finding ways to promote wider participation wherever there is a reservoir of returned participants.
• Assess the selection procedures of programs to see if behavior indicators can be used to identify individuals who will contribute back to their community

Major Actions Taken or That Will be Taken

• Bureau is looking at ways to increase alumni efforts and tracking.
• Future evaluations will use some of the indicators developed by the evaluators.
• Bureau reviewed use of behavior indicators and found them too costly and intrusive in the selection process for merit-based selection programs.

Title or Topic

Educational and Cultural Exchange Program s in Near East Asia and South Asia

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

ECA - Ted Kniker

Major Findings

• The program was given a PART rating of "Results Not Demonstrated."
• The program is managed well overall, but scored low on strategic planning for two reasons. First, the long-term goals are not set relative to an established baseline and do not have clear time frames and targets against which to measure annual progress. Second, the program does not have regional long-term goals for Near East Asia and South Asia, but rather worldwide, functional goals (e.g. Fulbright Visiting Student Program).
• The program scored high in purpose, planning and management.

Major Recommendations

• Adjust performance measures to include measurable timeframes and targets for achieving goals.
• Develop regionally-based performance measures.

Major Actions Taken or That Will be Taken

• Performance goals will be redrafted to include baseline and time frame language.
• Performance targets will be readjusted based on 2002 performance measurement.
• Analysis of performance will be conducted on several demographic variables including geography.
• A comprehensive performance measurement system will be developed to allow more effective annual measurement and reporting.

Strategic Goal 18

Human Resources

Title or Topic

Hardship Staffing Review

Evaluator

General Accounting Office

Department Contact

M/DGHR - Laura Hall

Major Findings

• Uneven burden of hardship staffing.
• Insufficient incentives for hardship staffing, especially due to twelve-percent pay cut overseas due to lack of locality pay.
• Strong disincentives for hardship service .

Major Recommendations

• Improve assignment and bidding data collection.
• Review assignment process to share burdens more fairly.
• Targeted recruitment.
• Develop financial and other incentives for hardship service.

Major Actions Taken or That Will be Taken

• Requested funding for upgraded personnel data systems.
• Strengthened fair share bidding program to place first those who have not recently served in hardship posts and to fill hardship posts first.
• Increased focus on recruitment of candidates with foreign language skills.
• Established working group to address both potential incentives and remedies for disincentives.

Title or Topic

Language Review

Evaluator

General Accounting Office

Department Contact

M/DGHR - Laura Hall

Major Findings

• Critical language gaps.
• Insufficient staff for adequate training.

Major Recommendations

• Strategic workforce planning approach to languages.

Major Actions Taken or That Will be Taken

• Targeted recruitment.
• Increased staffing to allow adequate training.
• Continued strategic workforce planning approach to determine language requirements of overseas positions.
• Increased attention to early development of strategic language skills needed in workforce.

Title or Topic

Bureau of Human Resources - Compliance Follow-Up Review

Evaluator

State Department Inspector General

Department Contact

M/DGHR - Laura Hall

Major Findings

• Bureau is appropriately addressing key human capital management areas.

Major Recommendations

• N/A

Major Actions Taken or That Will be Taken

• N/A

Title or Topic

Employee Satisfaction and Commitment Survey

Evaluator

State Department

Department Contact

M/DGHR - Laura Hall

Major Findings

• Employees are committed, public-service minded, and connected to the Department's mission.
• Employees want more focus on leadership and management.
• Retention is mostly based on struggles of families, particularly dual-career couples, in the mobile foreign service lifestyle.
• Lack of locality pay overseas erodes value of hardship differentials as incentive to service in hardship posts.

Major Recommendations

• N/A

Major Actions Taken or That Will be Taken

• Mandatory leadership and management training program for all mid-level employees and increased training at junior and senior levels.
• Revision of promotion precepts to emphasize leadership.
• Established spousal employment working group.

Title or Topic

Foreign Service Promotion System

Evaluator

State Department Inspector General

Department Contact

M/DGHR - Laura Hall

Major Findings

• Foreign Service "up-or-out" promotion system is appropriate and does not negatively impact Department's skill base or staffing levels.

Major Recommendations

• N/A

Major Actions Taken or That Will be Taken

• N/A

Strategic Goal 20

Infrastructure & Operations

Title or Topic

Embassy Security Construction Program

Evaluator

Office of Management and Budget Program Assessment Rating Tool (PART)

Department Contact

OBO - John Dudley or Robert Bell

Major Findings

• Program was given a PART rating of "Moderately Effective."
• OMB gave the Capital Security Construction Program the highest PART rating (78 percent) of any State Department program evaluated. It was found to have a clear purpose, used solid performance measures to measure results, and was effective in realizing program goals.

Major Recommendations

• Program has good planning, purpose and management, and enhancements to program should continue.

Major Actions Taken or That Will be Taken

• The FY 2005 Bureau Performance Plan will contain the specific long- and short-term goals and performance indicators identified in the PART, which were taken from the FY 2004 OBO Bureau Performance Plan.
• During the Bureau Performance Plan reviews with the Deputy Secretary, there will be an assessment of progress made on enhancing the Capital Security Construction Program.
• Fifteen capital construction projects are currently under construction, seven of which will be completed in FY 2003.