BCVision Core Support Organization Evaluation Shows Progress, Challenges Moving Forward

August 6, 2018

The following post is the BCVision Core Support Organization Evaluation Executive Summary completed by WeEval LLC. A link to the full report is included at the end of this post.

In 2014, local leaders and residents of Battle Creek were brought together by the W.K. Kellogg Foundation (WKKF) to discuss the varied economic conditions that existed in the city, and to identify how to improve economic conditions so that all residents could benefit. That convening resulted in the emergence of a shared vision for the community and an action plan that would guide efforts to transform Battle Creek into a vibrant and equitable place for all of its resident.

To help make that shared vision a reality, BCVision (BCV) was awarded a 5-year grant from the Kellogg Foundation in 2017 to guide a community-wide transformation effort toward a more vibrant and equitable Battle Creek. Under the work of the grant, BCVision identified three main areas of focus which served as the structural framework from which transformation efforts would emanate: talent development, job creation and a culture of vitality.

This document reports evaluation findings from Year One of implementation. It is guided by the following evaluation questions:

  1. To what extent have the Workgroups been supported?
  2. Does BC Backbone Support engage diverse communities across BC?
  3. Does the BC Vison promote equitable outcomes for BC residents?
  4. How does BC Vision communicate its objectives, activities, and successes?
  5. What evidence exists of long-term sustainability for the BC Vision?
  6. What are the lessons learned?

Process Evaluation: Interviews and Count Data

As a way of garnering perspectives from key individuals involved in supporting BCV’s work over the past several years, phone interviews were conducted with 21 non-staff who (a) represented a cross-section of sectors in Battle Creek, (b) served on myriad committees, and (c) served in various capacities on those committees. Those individuals brought with them years of experience and involvement in Battle Creek’s business and professional sectors and sought to offer their skills and knowledge to better the community that they lived and worked in.

Support Workgroups. When looking at the extent to which Workgroups have been supported in those efforts, interview data shows some disparities in responses. On the one hand, a number of interviewees felt that their skills, knowledge and abilities were not used sufficiently or in ways that was most advantageous to the project’s works. From a different perspective, a few interviewees suggested that their efforts through involvement in Workgroups were largely supported by both BCV and their organizations but noted that because of the complexities inherent in community revitalization and change efforts, support was oftentimes difficult to recognize or feel.

Organizational count data show that there were over 1,000 attendees at Workgroup sessions (CC&R, small business, workforce development, career pathways, steering committee, advisory committee, job systems, transportation, GED, co-chair, and combined workforce development and CC&R). There were 13 consultants and interns working in varying capacities to support the Workgroups.

Engaging Diverse Communities. As identified by a number of Workgroup members, various inroads were made into the engagement of diverse communities through the work of BCVision over the years. It was felt that through specific efforts promulgated by BCVision, significant engagements with and support of Battle Creek’s communities of color took place. In addition, it was noted that an important prerequisite to working effectively with Battle Creek’s diverse communities was internal diversity-related work that was done within BCVision and those supporting the work. That preparatory work was viewed as significant and essential to broader community engagement.

Organizational count data also reveals the engagement of diverse communities through BCVision. Specifically, this is captured through the number of minorities and women supported by the information from BCVision, and the number and type of meetings and interactions with the community.

Developing Equitable Outcomes. A review of the interview data revealed that there were several ways that equitable outcomes for residents were promoted through the work of BCVision over the years. One of the most notable was the notification of the 50-million-dollar grant award to Battle Creek Public School District resulting from the commissioned NYU study. A number of interviewees pointed to positive messaging which was associated with the purpose of the study, the findings and the actions taken by WKKF in impact issues surfaced through the study. It was felt that everything surrounding this outcome was a positive message about concrete efforts made to address identified inequities in education.

Another was the initial work with leaders and staff from various sectors in the community around diversity training. In that work, it was noted that BCVision used those trainings not only to set the foundation for deeper and more meaningful engagement by partners, but also to act as a mechanism to promote the goals and the work through the grant.

Effective Communication. In general, interview data showed that communication-focused support related to community engagement centered on creating and fine-tuning various communication channels in order to keep community members aware of the work through BCVision and encourage their involvement. Those channels included brochures, social media programs and working with print and broadcast media in communities of color. In addition, communication focused work by consultants and Workgroup members sought to develop more cohesive and streamlined methods and approaches to communicating information to the various racial and ethnic communities.

Analysis of interview data showed a feeling among many members that more communication of project activities should occur among members and across Workgroups. It was also felt that information flow across Workgroups needs to have more consistency and continuity, particularly as it pertained to connecting efforts to (a) BCVision’s broader vision and/or (b) action plans to guide the work.

Sustainability. Although not viewed as hard evidence, an overall analysis of interview responses highlights the deep sense of involvement in the project’s work by leaders and administrators of multiple sectors. Here, the length of involvement by many (some dating back to the beginning of the project), coupled with their expressed passion about the project’s vision, is seen as an important indicator of an ongoing commitment to the work, regardless of challenges experienced. In addition, the investment of time and resources by key stakeholders such as local businesses, city government, education affiliates and civic organizations is seen as a strong sign of a longstanding commitment to the city’s transformation through BCVision’s work.

In that the area of long-term sustainability is complex and require more extensive investigation and analysis of factors such as long-term commitments by key sector partners, this area will be analyzed in more depth in future evaluations.

Count Data Highlights

Support Workgroups: 1,046 attendances over 11 sub-committees, 13 interns and consultants.

Engage Diverse Communities: 112 volunteer meetings/interactions with community, 3 minority owned businesses were featured in Relocation Booklet.

Develop Opportunities: 111 organizations supporting BCVision’s Workforce Development initiatives, 34 businesses at job far, 15 education/employment events, 10 conferences/training.

Communicate Vision: 183 webpage visits, 4,000 Prep and Job Fair flyers distributed, 750 coloring books distributed, 3 quarterly news letters (166 sent in Oct. 2017, 148 sent in Jan. 2018 and 149 sent in April 2018).

Social Media Interactions: 297 shares, 117 comments, 2,909 reactions.

Gained: 128 followers, 117 page likes.

Sustain BCVision: 41 partner organizations.

Outcome Evaluation: Perception Surveys

Surveys were used to assess multiple aspects of this community development investment and provide perceptions of the long-term impacts. A survey was developed for use at events, conferences, and trainings to assess participant perceptions from various stakeholder groups. The survey included background questions (gender, race, length of time in Battle Creek) that allow for a consideration of variation across multiple sectors of Battle Creek.

For this 2017-18 report, surveys came from multiple administrations (total sample=190). A core sample was comprised of steering committee, Workgroups, volunteers, consultants, and interns (n=20). A short survey was also conducted at the 2018 Cereal Fest (n=97) and the Juneteenth Celebration (n=73). These data provide descriptives and a baseline for future analyses. Individuals differed in gender, with 63.3% female and 36.1% male. Individuals also varied in racial identification, with 52.6% White and 47.4% non-White respondents. People had lived in Battle Creek for different lengths of time: 30.8% had been here 10 years or less, 20.5% for 10-20 years, and 42.2% for over 20 years.

Responses to survey prompts revealed initial perceptions for the three long-term objectives for community development in Battle Creek: talent development, job creation and culture of vitality. 54.2% of respondents agreed or strongly agreed that Battle Creek prepares people to be both school and career ready. The most positive perceptions were reported for questions related to a culture of vitality: 80% agreed or strongly agreed that Battle Creek can change with the times, and 68.6% felt that Battle Creek is moving in the right direction.

Perceptions of Project Long-Term Outcomes
Education and School Readiness:

–     BC prepares young people to be college ready

54.2 % of respondents agreed or strongly agreed
Jobs and Career Training:

–     BC prepares people to be career ready

54.2 % of respondents agreed or strongly agreed
Culture of Vitality:

–       BC can change with the times

–       BC is moving in the right direction

80 % of respondents agreed or strongly agreed

68.6 % of respondents agreed or strongly agreed

There were additional survey prompts that directly assessed the role of BCVision. 51.1% agreed or strongly agreed to the prompt “I know about the work of BCVision.” 68.6% reported that BCVision was promoting positive community development.

Perceptions of BCVision
Awareness and Visibility: I know about the work of BCVision 51.1 % of respondents agreed or strongly agreed
Action and Potential: BCVision is promoting positive community development 68.6 % of respondents agreed or strongly agreed

These are promising findings, and important indicators to track over time. These data provide a baseline for future comparisons as the work of BCVision grows in scope and scale. Larger sample sizes gained through future survey administrations will allow for examinations of heterogeneity across groups of respondents.

Notable High-Level Outcomes

This evaluation report synthesizes available data to describe first year implementation of BCVision. The process evaluation findings describe participants’ experiences as they interact with the activities and mechanisms of BCVision. The outcome evaluation findings present initial perceptions of the long-term goals of the project and notable outcomes during Year One. As a complex intervention in community development and thriving it is important to track core interests over time, and these findings will guide as a basis of comparison for future analyses.

As a result of the overall work of the project to date, several notable high-level outcomes are captured and summarized in the Table below:

Notable Outcomes Description of Outcomes
Cross-Sector Involvement Continuous involvement from individuals representing multiple sectors germane to the work including community-based organizations, large and small business, education, city government, social services.
Inroads into Engaging Diverse Communities Important developmental work occurred on (a) promotion of an equity lens in education and employment (b) implementation of mentoring programs in support of women of color, and (c) knowledge building on diversity and equity in institutions, organizations and city agencies in Battle Creek.
 

Learning and knowledge Building for Informed Decision-Making

The use of knowledge acquired from (a) rigorous scientific studies and (b) on the ground learning to inform decision making (e.g. grant funding to Battle Creek Public Schools).
Building a Roadmap Germane to Battle Creek’s Success Important learning, relationship building, and developmental work occurred as an essential part of community transformation that involves (a) citizens from all of Battle Creek’s diverse communities and (b) critical sectors which comprise Battle Creek’s business, education and social infrastructure.

Summary of Successes, Challenges and Learnings 

Interviews with Workgroup members captured a plethora of insights and perspectives that covered work that was specific to the Backbone Core Support project as well as work more germane to the broader BCVision initiative being advanced by cross-sector partners. Following are identified successes, challenges and lessons learned, broken out by Backbone Core Support work and BCVision initiative work.

Summary of Successes

Backbone Core Support Successes Initiative Successes
Convening Brought together multiple cross- sector groups in Battle Creek to work toward a common purpose and shared vision for community revitalization under the guidance of BCVision. Implementation of diversity and equity strategies Promoted, fertilized and propagated (a) diversity and equity conversations across sectors and in communities around the city and (b) support of work to deepen diversity and equity concepts and practices in myriad organizations and businesses in Battle Creek.
Community engagement Fostered beginning steps on community engagement between Battle Creek’s large and small employers and communities of color on (a) employment    opportunities,    (b)  workforce preparedness, and (c) workforce development. Collaboration Contributed to a more focused effort on workforce development and college and career readiness with cross-sector input and involvement.
Communication Engaged in information gathering and other developmental work needed to spotlight and support business development in Battle Creek. Gap analysis – Members beginning to surface and explore variations in challenges experienced in business community and setting the stage for deeper exploration and problem solving.
Equity Engaged in information gathering and other developmental work needed to support equity in education in the area’s schools and in education outcomes of the city’s low performing students.

Summary of Challenges

Backbone Core Support Challenges Initiative Challenges
Fostering the understanding of BCVision Need to define key concepts that undergird the work, creating a common language and understanding by all who are (a) involved in supporting and actuating the work and their roles, (b) being exposed to various efforts, and (c) benefactors of efforts. Need for more of an action orientation Converting more conceptual conversations that occur in many Workgroups to tangible, measurable actions on the ground.
Maintaining a balanced representation of member involvement Need to ensure the maintenance of balanced representation and thus voices and perspectives from all communities. From the sector side, that includes balancing involvement from small businesses and non-grant funded companies and organizations which may have different foci or interest. From the community side, that includes a representation of voices from different strata and/or experiences within racial and ethnic communities. Connecting and sharing work, knowledge and information across Workgroups The lack of sharing streams of work across Workgroups make it difficult for members to connect activities from one Workgroup to another and how overall efforts relate to the broader vision being advanced through BCVision.
Meeting coordination Coordination of meeting times and locations so as to optimize involvement by diverse cross-sections of the community.

Summary of Learnings

Learnings for the Backbone Core Support Learning for BCVision Initiative
Learning from the experience Learning from the experience and maintaining an understanding of the ways in which the work impacts the community from the work advanced through the Backbone Core Support. Learning from the experience Learning from the experience and maintaining an understanding of the ways in which the work impacts the community from the actions taken by Workgroups and from feedback received from the community.
Continuously assessing ways to continuously improve Constantly assess ways to improve and advance the work actuated through BCVision’s Core Support through input from all stakeholder groups. Continuously assessing ways to continuously improve Identifying where and how intracommunity resources and expertise  are  helping  to  support  and   reach desired economic    outcomes, and when resources and expertise need to be sought outside of the community.
Trumpeting victories The importance of finding and talking about victories, not only viewing    the    community    from    a    deficit perspective. The importance of large business and institution    involvement    –    The    visible presence of large businesses and community organizations/institutions  involved in BCVision’s work help to champion the efforts while enhancing    broad-based community engagement and support.

Recommendations

Based on the findings, summary and conclusions of this report, the following recommendations are made for BCV and the Core Backbone Support efforts:

  1. Continue to improve the communication and marketing efforts such as breaking down the silos between Workgroups, reinforcing visibility and brand recognition of BCVision, and enlisting partner organizations in the promotion of BCVision awareness
  2. Create a balanced representation in sector involvement
  3. Revisit goals and strategies related to community engagement
  4. Build from initial successes in growing partnerships and capacities for community development.

A full copy of the report can be read by clicking here.