Public Use of the Open Data Portal
Open data portals are a centralized platform which are intended to provide citizens and government with a singular location to search and locate authoritative open data from government entities. The following sections describe how users can interact with open data portals.
Open data portals offers several different ways to identify and locate data.
The first and most comprehensive way to find data is the search bar. The search bar scans through the titles, descriptions, and tags/keywords (these are all metadata) to locate any matches to the input.
If the user has a broader subject, but maybe not a specific term, the topics section breaks the returns into themes like water, health, or transportation.
The groups area filters returns based upon the organization that published the data. This can be helpful if you want to see a specific agency’s data (i.e., Department of Water Resources vs. State Water Resources Control Board).
Once a dataset has been identified and located, the user will be presented with various sets of information about that dataset. A dataset is not a single file but container for associated files (resources).
First is a description of the dataset (a dataset can have multiple files or resources associated with it). This will provide the user with an overview about what the data contains and possibly other details such as the source, use of the data and links for further information.
Next are the resources (specific files) associated with the dataset. These can be in a multitude of different file types but comma separated value (csv) files are the most prevalent. Generally the user is presented with an option to preview the data or download it.
The metadata (data about the dataset) are usually listed as well. These include topics such as:
- Last modified date
- Release date (first publication)
- Frequency of expected updates
- Links to pages with more information about the dataset
- The unique identification number for the dataset
- Any licenses associated with the dataset (generally Public Domain)
- Author of the data (this can be different from the publisher)
- Contact name and email
Organization or Group
The organization, group, or publisher of the data is usually listed along with a brief description about the organization.
Any topics associated with the dataset are generally listed.
Tags or Keywords
Any tags or keywords associated with the dataset are also usually listed.
Datasets are containers for resource files. These can include file types such as:
- CSV – Comma Separated Values
- XLS – MS Excel file extension
- XML – Extensible Markup Language
- RDF – Resource Description Framework
- ODF – Open Document Format
- ODS – Open Document Store
- TSV – Tab Separated Values
- Zipped Shapefile – A shapefile is actually a collection of several files with the same file name, but differing extensions. For the Open Data Portal, each shapefile should contain (at a minimum) the following files:
- SHP – defines the vertices of the shape
- DBF – defines the attribute table
- PRJ – defines the projection for the data
- SHX – shape indexing file, for efficient processing
- REST Map/Feature Services – Representational State Transfer
- KML – Keyhole Markup Language
- KMZ – Zipped Keyhole Markup Language
- HTML/URL – Hypertext Markup Language/Uniform Resource Locator
- DOC/DOCX – Microsoft Document
- TXT – Text
- JPG – Joint Photographic Experts Group
- PNG – Portable Network Graphics
- GIF – Graphics Interchange Format
- TIFF – Tagged Image File Format
- PDF – Portable Document Format
- ZIP – Compressed File
- ODT – Open Document
- BIN – Binary File
For each of these file types the user will usually be presented with one or more of three options:
Opens the resource preview page
Downloads a copy of the file
Opens a link to another site containing the data
The resource preview page can provide many different types of previews from tables, maps, and charts to iframes showing pdf documents and external web pages. There may be an additional description specifically for the resource under the title as well as a link for downloading the data or visiting the referenced web page directly. There will usually be additional metadata specific to the resource. This will include fields such as the file type, file size, resource type and last modified timestamp.
Due to CSV files being the prefered machine readable file type for open data portals, users will generally be presented with a table that shows records associated with the data file. CSV files are the only file type that can present this type of preview as they have been loaded into a backend database that provides preview and API functionality for the data table.
Some open data portals provide a “Filter” button. This allows users to limit the data shown by selecting a field within the data table and adding a filter type of either a value, range of values or geographic distance from a defined location (using latitude and longitudinal values). This can help users identify if the data table they are previewing contains the data values they are looking for.
If available, graph previews generally allow the user to select from various graph types. Once the type of graph is selected the user can associate Axis 1 and Axis 2 with any of the fields in the data table.
Map previews allow the user to preview the data overlaid on an underlying basemap. Not all open data portals provide this but many do. This is only possible if the data contains Latitude and Longitude fields or a GeoJSON field.
API (Application Programing Interface)
CSV files that have been loaded into the portals backend database can also provide API access if the portal allows for it. APIs allow for web based applications to query the data in the table for specific information. APIs are a bit like a menu. They let third party applications know what can be queried in the table, how to ask for it and whether the returned data will be formatted in the proper way to interface with their application. The API query is returned in either a JSON or XML format depending on the parameters in the query.
The Government Operations Agency is focused on making open data, public data accessible by anyone that can be freely shared and redistributed, available through online means. Open data is open to the public and encourages collaboration and community participation by cultivating it for better outcomes. Through civic engagement, individuals are able to improve the quality of life and help government go where it cannot go alone. Engagement can influence policy and provide data-driven decisions. To find more information about civic engagement events please select the “Civic Engagement” button near the top of the page (https://findanewway.ca.gov/).
What is Civic Engagement?
Civic engagement involves the use of open data and identifying how this information can bring awareness to improve processes, programs, and/or policies. Civic engagement is about taking the data further. Lists of datasets do not actively engage public members nor do they show the core value they possess. To get the most from data, one must bring it to life and capture its meaning and full potential through the development of apps, visualizations, dashboards, or other tools. Improvement is based upon partnerships stimulating innovation that will ultimately shape programs and services, help deliver input, and advance public policy.
Civic engagement is bringing the right people together to create projects that best address problems governments face. There are three essential members to include when developing events including technological experts, subject matter experts, and front line community members that are pieces to the larger puzzle of civic engagement.
- Technological Professional/Coders– Volunteers eager to contribute to government through their expertise but have little to no idea what to build from the data available. They can develop apps and visualizations, but lack understanding about everyday issues people encounter.
- Subject Matter Expert/Government Staff– Most knowledgeable in the particular field and have insight as to what should be created. They will know what to build and what is needed to drive effective change.
- Community Member/Front Line Participant– People that encounter issues and want to share insights around these challenges. They can identify a problem that needs to be resolved. It is important to remember building with and not for.
Why is Civic Engagement needed?
With civic engagement we have the ability to bring innovators across the State of California that can leverage open data and help community and policymakers. Government is able to break down silos and work with the community to utilize existing technology for better operations and solutions to meet the public’s needs. Civic engagement is about connecting groups that otherwise would not be talking to each other. Opportunities for participation in public service are created while new relationships and networks are formed. At its core, government is about doing collectively what cannot be done alone and this is why civic engagement is vital.
Through civic engagement products that succeed are those that best support work in which groups and people are already committed to. This allows support for an open and involved culture which empowers the groups and people to create change for themselves. Civic engagement is not always about saving lives, rather it is about making government function better. It is about finding new methods which improve everyday activities.
For more information about the California Open Data Portal and other open data resources please visit CDT’s Documentation page: https://data.ca.gov/pages/documentation