Virtual Exhibits Investment Program - Program Guidelines : Evaluation Process

Proposals for VMC investment greatly exceed the program's budget in any given round.

The VMC Investment Program staff assess all eligible proposals to ensure compliance with VMC requirements including the mandatory criteria identified below. Proposals that do not meet the mandatory criteria are immediately eliminated from consideration. The VMC Secretariat then prepares summary information of all remaining proposals to assist the VMC Editorial Board in its evaluation process.

The VMC Editorial Board's evaluation process entails individual assessments and group discussion. A decision regarding the proposals that are recommended for VMC investment is made at the annual VMC Editorial Board meeting.

The evaluation process spans approximately four months from beginning to end, including the VMC Editorial Board meeting.

VMC Editorial Board

The role of the VMC Editorial Board includes the evaluation of proposals submitted to the Virtual Exhibits Investment Program, taking into account the desired program outcomes, the Virtual Museum of Canada operating principles and content policy, and the specific criteria that are set out below. The Board provides recommendations to the Director General of CHIN with respect to the proposals in which the VMC should invest.

CHIN executes contracts with museums to carry out the proposed productions as soon as the recommendations have been received from the Editorial Board. For reasons of public accountability, CHIN reserves the right to reject Editorial Board recommendations.

The VMC Editorial Board is composed of:

• A chairperson;
• 10-12 members representing diverse expertise (representatives from the museum community, as well as representatives from other relevant sectors, e.g. education, publishing, broadcasting, new media).

Evaluation Criteria

The criteria identified below are considered in the evaluation process. Certain criteria are mandatory; others represent factors that are taken into account as the Editorial Board selects among competing proposals. The Board reserves the right to refine the criteria as the Program evolves.

Even though content is located on websites across the country, the VMC is being presented to the public as a collective entity. Proposals are therefore evaluated not only on their own merits but also within the overall context of other existing and proposed content to ensure balance among disciplines and themes, approaches to the presentation of content, and target audiences.

Mandatory Criteria

General

Proposals must include details for the creation of the following online products as a cohesive production on the same subject or topic:

• An exhibit (or interactive resource)
• A learning object collection (consisting of no less that 5 learning objects)
• A lesson plan

Any proposal which does not include all of the above components will be removed from further consideration.

All products must be created in both official languages and be available for a period of five years following their launch. Any proposal which does not include evidence of measures to meet these obligations may be eliminated from further consideration.

Specific

• Clearly identified target audience(s) for the proposed content.
• Specific measures to conduct audience evaluations during the development of the online production.
• A comprehensive production plan which covers the development of the entire product (i.e. exhibit or interactive resource, associated learning object collection and associated lesson plan) and which identifies the duration of each activity.
• Contribution to the project by the institution and its partners. Note: this contribution does not need to be financial. See Annex A - Investment by Museums for examples.
• Direct involvement of an accredited educational expert during the development of the learning object collection and lesson plan(s).
• Each learning object must contain:
  • a minimum of at least one digital asset;
  • at least one learning objective;
  • clearly identified curricular links.
• Lesson plan(s) must directly involve the use of the learning object collection identified within the same proposal.

Rated Criteria

Primary Criterion

• Demonstrates a dynamic exploration of Canada's diverse heritage in ways that encourage enjoyment and learning. - 30 points maximum

  Very well - 30 points:
  • A topic that would appeal to a broad spectrum of Canadians that presents little known information.
  • A topic that is worthy of wider dissemination. This would include subject matter related to school curricula, significant historical events, tourism, and current events or issues.
  • Treatment and organization of the content that is appropriate to the identified target audience. The proposal demonstrates a creative/original approach to presenting the subject matter. That is to say that the proposed product involves the generation of new ideas or concepts, or new associations with museum content that represents a theme/concept or topic in a new and novel way.
  • Demonstrates a dynamic exploration of Canada's diverse heritage in ways that encourage enjoyment and learning.
  • Audience engagement is a key element of the concept through both interactive and social media elements.
  Well - 20 points:
  • A topic that would appeal to a wide spectrum of Canadians. This would include subject matter related to school curricula, significant historical events, tourism, and current events or issues.
  • Content provided comes from a variety of sources, images, text, audio and video.
  • Treatment and organization of the content is appropriate to the target audience identified and demonstrates some creativity in approach. Audience engagement is encouraged.
  • Demonstrates a dynamic exploration of Canada's diverse heritage in ways that encourage enjoyment and learning
  Somewhat - 10 points:
  • A topic of some relevance to a broad spectrum of Canadians; Treatment and organization of the content demonstrates little creativity or means for audience engagement.
  Not at all - 0 points:
  • A topic of very little appeal or relevance to a broad spectrum of Canadians; Proposed treatment and organization of the content is uninspired.

Secondary Criteria

• Exploits the digital medium to create experiences that are not feasible in physical space or by using analogue media (e.g., through presentation techniques; or, through assembling collections that do not co-exist in physical space). - 20 points maximum

  Very well - 20 points:
  • Production clearly supports and enhances the identified communication objectives.
  • The specific technologies to be used are clearly identified, as well as how they will be used to enhance the presentation of the content and the overall user experience to the extent that it will stimulate users to respond to the content.
  • Production presents objects in a manner which is not feasible in physical space using technologies that are adapted to the subject matter (GPS, spatial data).
  Well - 15 points:
  • Production supports the identified communication objectives.
  • The technologies to be used are identified in general terms and a high level description of their use in the presentation of the content is provided.
  • Production presents objects in a manner which is not feasible in physical space using technologies.
  Somewhat - 5 points:
  • Production somewhat supports the identified communication objectives, although the technologies to be used and their application to enhance the presentation of the content and the overall user experience with the content is not clearly presented.
  • Production does not clearly indicate that it will present objects in a manner which is not feasible in physical space.
  Not at all - 0 points:
  • Production does not clearly support or enhance the identified communication objectives. Identification and application of specific technologies are not clear as to how they will enhance the presentation of the content or the overall user experience, in such a way as to stimulate users to respond to the content.
  • Production does not present objects in a manner which is not feasible in physical space.
• Provides value for money. (N.B. The ceiling for investment is $400,000, including GST/HST. - 25 points maximum

  A concept associated with a comparison of the investment against the value of the output and a qualitative and quantitative judgment over the manner in which the resources involved have been utilized and managed.

  Very well - 25 points:
  • The proposal offers a high ratio of contributions in terms of content and design elements from the lead institution, partners and participants in relation to the investment requested.
  • Contributions include most of the following: staff time and expertise, copyrighted materials, artefact/specimen related information, research and writing, as well as the management, coordination and marketing associated with the project and outreach activities.
  • Resources have been allocated to most aspects of the project including quality control for both technical and content aspects of the product in both official languages.
  • Production provides skills development and capacity building opportunities for the institution.
  Well - 20 points:
  • The proposal demonstrates a moderate ratio of contributions in terms of content and design elements from the lead institution, partners and participants in relation to the investment requested.
  • Contributions include a few of the following: staff time and expertise, copyrighted materials, artefact/specimen related information, as well as the management, coordination and marketing associated with the project.
  • Resources have been allocated to aspects of the project including quality control for both technical and content aspects of the product in both official languages.
  • Production provides skills development or capacity building opportunities for the institution.
  Somewhat - 15 points:
  • The proposal demonstrates a low ratio of contributions in terms of content and design elements from the lead institution, partners and participants in relation to the investment requested.
  • Contributions in only one or two of the following: staff time and expertise, copyrighted materials, artefact/specimen related information, as well as management, coordination and marketing associated with the project.
  • Resources have not been allocated to certain aspects of the project including quality control for both technical and content aspects of the product in both languages.
  Not at all - 0 points:
  • The budget for the proposed product is not in keeping with the perceived value of the finished product in terms of content sources and design elements.
  • The ratio for management/operating costs is high in relation to low investment on the part of the lead institution and its partners.
• Involve partnerships³ with other public and private organizations for content development, educational expertise, and/or technical expertise. - 10 points maximum

  For the purposes of the VMC Virtual Exhibits Program, partnership is defined as meaning shared risk and ownership, through contribution of expertise and/or collections.

  Very well - 10 points:
  • Production involves two or more museums and/or organizations as partners contributing content and/or expertise, as well as direct involvement in the development of the overall production, which expands and enhances the breadth, depth and credibility of the content.
  Well - 7 points:
  • Production involves two or more museums and/or organizations as partners contributing content and/or expertise which expands and/or enhances the breadth and depth of the content.
  Somewhat - 3 points:
  • Production involves one museum and/or organization as a partner contributing content and/or expertise which expands and/or enhances the breadth and depth of the content.
  Not at all - 0 points:
  • Production involves no other museums and/or organizations as partners.

• Proposed learning object collection and lesson plan(s) contain clearly identified curriculum to which the learning objects relate, as well as, clearly defined learning objectives. - 10 points maximum

  Very well - 10 points:
  • Curriculum related to the proposed learning object collection and lesson plan is clearly identified (age ranges, subjects, provinces).
  • The learning objectives are clearly stated and include reference to the educational outcomes.
  • Description clearly identifies the type and number of digital assets to be included in the learning object collection.
  • Learning object collection and lesson plan apply to one or more provinces.
  Well - 7 points:
  • Curriculum related to the proposed learning object collection is identified but not detailed and may be missing elements (ranges, subjects, provinces).
  • The learning objectives are stated.
  • Description identifies the type of digital assets to be included, but not necessarily the number to be included in the learning object collection.
  • Learning object collection and lesson plan apply to one province.
  Somewhat - 3 points:
  • Curriculum related to the proposed learning object collection is identified in very general terms but lacks specificity (no ranges, subjects, provinces identified).
  • The learning objectives are not clear.
  • Type and number of digital assets is not clearly indicated.
  Not at all - 0 points:
  • Information provided is too general or insufficient in clarity to determine the exact curriculum to which the learning object collection and lesson plan relate.
  • Little or no detail about learning objectives provided.
  • Number and type of digital assets is not detailed or missing.

• Involve youth in the creation of content. - 5 points maximum

  Very well - 5 points:
  • Production involves youth in an active role in the conceptual and technical development of the production, as well as in a passive role (focus groups).
  Well - 3 points:
  • Production involves youth in an active role in the conceptual or technical development of the production, as well as in passive role (focus groups).
  Somewhat - 1 point:
  • Production involves youth in a passive role only (focus groups).
  Not at all - 0 points:
  • No youth involvement.

Next Section

---

³ All partners must commit to their participation by means of a signed partnership letter, which must clearly specify what the partner will be contributing to the project. The signed partnership letter will:

• establish the duration of the partnership;
• confirm the amount of any cash contribution;
• describe any in-kind support and confirm its value;
• identify the specific content/collection(s) that they are providing access to and/or explain the expertise or services that will be provided.

NOTE: For the purposes of the VMC Virtual Exhibits Program, partnership is defined as meaning shared risk and ownership, through contribution of expertise and/or collections.