Additional Details on FY 2003 Performance Results - Strategic Goal 11: Public Diplomacy and Public Affairs

FY 2003 Performance and Accountability Report
Bureau of Resource Management
December 2003
Report

Performance Goal 1: International Public Opinion

Initiative/Program #1: Reaching Out to Allies and Regional Powers

PERFORMANCE INDICATOR #1: LEVEL OF INTERNATIONAL PUBLIC UNDERSTANDING OF U.S. POLICIES
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
Baseline: Evidence shows that information provided has reached intended user. N/A

Speakers: Sent 846 speakers abroad, of which 168 traveled to the Middle East and South Asia.

Digital Video Conferences: The Speakers program supplemented by 480 digital video and telephone conferences.

Publications: Through Embassies worldwide, distributed over 500,000 copies of 16 new publications, approximately half of which dealt with counterterrorism or Iraq.

On Target
Details of 2003 Results
  • Steps Being Taken to Improve Performance: The Department is conducting workshops and putting into place a system of incentives and disincentives to encourage greater and more appropriate use of the RESULTS database. The Department is also looking to alternative means for assessing specific program initiatives.
  • Data Availability: Impact data was drawn from Embassy reporting into the RESULTS database, and from a May 2000 Field Survey of Public Diplomacy Programs.
  • Data Reliability: The numeric output data came from the Tracker Database, and the anecdotal impact reporting came from the RESULTS database. Both are deemed reliable sources for this type of information.
  • Data Validation: Anecdotal reporting from field officers indicates that the programs are successful in disseminating information about U.S. policies to target audiences.
  • Partners: Speakers were drawn from throughout the U.S. Government, think tanks, NGOs and the academic community.
PERFORMANCE INDICATOR #2: NUMBER OF PAGE VIEWS TO DEPARTMENT'S
INTERNATIONAL WEBSITE, MISSION WEBSITES AND LISTSERVS
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
120,000 unique weekly users, 8,000 subscribers to listserv. A total of 120,000 average daily pages read, 8,000 subscribers to listserv. Approximately 10% of users come for reference material on U.S. history, government. The USINFO website was rated by users as "above average" in customer satisfaction, slightly higher than what users give commercial websites. 10,153 subscribers to listservs targeting all of the geographic regions of the world in six languages. Above Target
Details of 2003 Results
  • Reason for Target Revision: We have changed from counting "unique weekly users" to counting "average daily pages read." This is a more accurate tally.
  • Data Availability: Although current web usage analysis software is good, we are further refining our web usage counts for USINFO by adopting an improved web analysis software package - Webtrends.
  • Data Reliability: Current web analysis software meets the minimum standards for accuracy of results.
  • Data Validation: Since the audiences for both the websites and the listservs are self-selecting, we have a high degree of confidence that information imparted in this manner is reaching the intended users.
  • Partners: The contents of the websites and listservs were coordinated with all the appropriate government sources, including the White House, DOD, DOJ, USAID, etc.
PERFORMANCE INDICATOR #3: THE LEVEL OF MEDIA PLACEMENT IN FOREIGN MARKETS IN BROADCAST AND PRINT
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
Baseline: Evidence shows that information has reached intended user. N/A

Placement of Broadcast Media: Two short documentary films aired in 25 countries (Rebuilding Afghanistan & Afghan Spring); 120 special TV productions; 75 Foreign Press Center Briefings; 31 TV Co-ops with foreign broadcasters; 4 co-productions for Russian Public TV, Belarus TV, Georgian TV, and French African TV, 184 TV interviews; and 38 radio interviews. Print Media: 22 FPC briefings, and 87 interviews.

Actual op-ed/by-liner placement, by region: AF - 53; EAP-60; EUR 434; NEA-43; SA-78; WHA-238.

On Target
Details of 2003 Results
  • Data Availability: 1) Embassies are often understaffed to monitor and record all placements - particularly electronic media placements. 2) Lack of financial resources has prevented the Department from conducting full-scale evaluations of impact. A limited media trends analysis on highly defined topics is available from our own in-house Media Reaction capability.
  • Data Reliability: Extant placement data was captured in the RESULTS database and in a PA operated tracking database. The data we have is reliable, but provides an incomplete picture of our full placement, due to inadequate monitoring and reporting at the field level.
  • Data Validation: Foreign media placements reflect, ipso facto, that USG policy material reached foreign audiences.
  • Partners: DoS worked close cooperation with IBB to convey DoS television programming over the IBB satellite system.
  • Other Issues: Television placements listed are FY 2003 new benchmarks, to be used to gauge effectiveness in media placements in future years.

 

Performance Goal 2: Mutual Understanding

Initiative/Program #2: Engaging Audiences More Deeply

PERFORMANCE INDICATOR #1: PERCENTAGE OF PARTICIPANTS WHO INCREASED THEIR
UNDERSTANDING OF THE HOST COUNTRY IMMEDIATELY AFTER THEIR PROGRAM
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
92% N/A 89% On Target
Details of 2003 Results
  • Data Availability: The data collection for 2003 is complete. Three data issues emerged: Some of the data was collected during the launch of the war with Iraq and we received a slightly lower response rate than expected. Part of the sample for 2003 involved a program that brings parliamentarians to the United States for 10 to 14 days with a focus on teaching about U.S. government structure. Cultural understanding is a lower priority of this program, which somewhat reflects the lower performance result. Resources for evaluations are limited, which affects the depth of the evaluations. One example is that we use a retrospective recall methodology instead of pre-program and post-program comparisons, which is a stronger methodology for measuring change of understanding. Current evaluation resources also do not allow us to evaluate in every country or region.
  • Data Reliability: The data is reliable. Independent, external evaluators generated statistically valid samples. In 2003, 4,471 former program participants from 1993 - 2002 were surveyed with 2,782 responding for a response rate of 62 percent. Respondents represented a broad cross-section of exchange programs. Program managers can make sound decisions based on the findings from the data findings. Results are based on survey responses with a plus or minus 3 percentage points margin of error. Within this range of the target, the FY 2003 result is considered "On Target."
  • Data Validation: The data is useful because it reflects the fundamental goal of ECA programs: to increase mutual understanding. Achievement of the target indicates effectiveness of ECA programs.
  • Partners: The results were achieved with the assistance of over 100 non-profit organizations, the U.S. academic community, and over 80,000 American volunteers.
PERFORMANCE INDICATOR #2: PERCENTAGE OF PARTICIPANTS WHO REMAIN IN CONTACT WITH HOST COUNTRY PEOPLE MET ON THEIR PROGRAM ONE YEAR OR LONGER AFTER THEIR PROGRAM
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
75% N/A 81% Above Target
Details of 2003 Results
  • Data Availability: The data collection for 2003 is complete. Three data issues emerged: Some of the data was collected during the launch of the war with Iraq and we received a slightly lower response rate than expected; yet it was still well in the majority of potential respondents. Part of the sample for 2003 involved Americans who host visitors or who act as a resource, generally for one to two hours. To track longer-term results, up-to-date contact information for former participants is necessary. Currently resources do not allow for the continued tracking and engagement of alumni to the extent needed to reach critical masses of alumni in various regions.
  • Data Reliability: The data is reliable. Independent, external evaluators generated statistically valid samples. In 2003, 4,471 former program participants from 1993 - 2002 were surveyed with 2,782 responding for a response rate of 62 percent. Respondents represented a broad cross-section of exchange programs. Program managers can make sound decisions based on the findings from the data findings.
  • Data Validation: The data is a valid indicator because if reflects the primary goal of exchange programs, building and maintaining relationships. The percent of participants who remain in contact demonstrates that exchange programs foster personal and professional linkages that form a foundation of trust to engage other countries on short-term policy issues as well as establishing long-term partnerships, thereby producing stronger international relations for the United States.
  • Partners: The results were achieved with the assistance of over 100 non-profit organizations, the U.S. academic community, and over 80,000 American volunteers.

 

Performance Goal 3: American Values Respected Abroad

Initiative/Program #3: Communicate More Effectively With Global Publics

PERFORMANCE INDICATOR #1: TARGETED PUBLICS RECEIVE INTENDED MESSAGE
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
Baseline: Evidence shows that information provided has reached intended user. N/A

hi Magazine: The first three editions (totaling 150,000 copies) of "Hi" magazine, in Arabic, were issued and distributed throughout the Middle East. Production deadlines met.

American Corners: Established 23 new American Corners. There are now 79 American Corners distributed as follows: EUR/NIS: 56; AF - 18; EAP: 4; SA.

Persian Website: The U.S. has no official presence in Iran, so the USINFO Persian website serves as a virtual Embassy and cultural center for thousands of Iranians who have access to the Internet but do not read English.

On Target
Details of 2003 Results
  • Data Availability: Although monitoring and evaluation is planned for each of these initiatives, they are too new for us to be able to collect much useful data as yet.
  • Data Reliability: The anecdotal and output data listed above came from Embassies and from our RESULTS database, and is highly reliable.
  • Data Validation: Although as yet scant, the anecdotal data provided above is valid in that it speaks to the effectiveness of these new initiatives.

 

Performance Goal 4: Domestic Understanding of Foreign Policy

Initiative/Program #4: Outreach to Expanded U.S. Audience

PERFORMANCE INDICATOR #1: NUMBER OF INTERVIEWS AND CONTACTS WITH U S. MEDIA
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
16,000 contacts/year. N/A Baseline: 120 daily press briefings; 15,000 calls from the press; 1,000 print, radio, and TV interviews, Opinion Editorials (Op-Eds) and major press conferences with Department principals. On Target
Details of 2003 Results
  • Data Reliability: Data collected because it is entered into database for media events, and telephone calls are tracked through an automated system.
PERFORMANCE INDICATOR #2: INCREASE IN THE NUMBER OF OUTREACH ACTIVITIES TO TARGETED U.S. AUDIENCES
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
Evidence shows that information provided has reached intended user. N/A Baseline: Distributed 14,000 curriculum video packages to U.S. educators; conducted over 1500 outreach activities; reached over 12,000 students through in-house briefings and other programs; conducted 23 student town meetings at high schools and colleges; conducted over 600 Washington and regional events for Department's speakers program; and delivered over 70 presentations at State and National Governmental conferences. On Target
Details of 2003 Results
  • Data Reliability: Data collected because it is entered into database for media events, and telephone calls are tracked through an automated system.
PERFORMANCE INDICATOR #3: INCREASE IN "HITS" ON THE DEPARTMENT'S DOMESTIC WEBSITE
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
4.5 million hits per month. N/A 5 million hits per month. On Target
Details of 2003 Results
  • Data Reliability: Data collected through Webtrends software.

 

Initiative/Program #5: Historical Research and Publications

PERFORMANCE INDICATOR #4: NUMBER OF HISTORICAL, RESEARCH, AND EDUCATIONAL PUBLICATIONS
Initial FY 2003 Target Revised Target FY 2003 Result FY 2003 Performance Ratings
Make progress in meeting mandated compliance. Publish four (4) FRUS volumes. Baseline: Published six (6) Foreign Relations of the United States (FRUS) volumes. Above Target
Details of 2003 Results
  • Reason for Target Revision: Target changed to become quantifiable.
  • Partners: Central Intelligence Agency.