Method Evaluation

To strengthen confidence in survey findings is it is important to evaluate both the method and data gathered from the Internet survey. Survey data was triangulated with feedback from monthly Charlestown Planning Commission Openspace Workshops and a follow-up Internet survey for respondents who provided an e-mail address while taking the survey.
Click here to view the actual follow-up survey.

Below are seven issues of concern or comment, raised by the 30 respondents from the follow-up survey. Respondent comments follow a brief discussion of the issue, and are bulleted.

Survey Site Selection

One respondent raised a concern about which survey sites were selected. As mentioned in the limitations section, the process by which sites were selected could have been more democratic and comprehensive, had the Planning Commission understood and supported a more thorough selection process. This could have also been facilitated at a Town Meeting at which more residents attend and greater public input could have been gathered at this point in the project. However, it is important to remember that this is an iterative process and the 23 sites included in the survey do not represent the final options or choices for potential openspace in Charlestown.

• Participants at the original meeting, when asked to pinpoint suggested spots, should have had more time to work. Like come back next week with suggestions. To the best of my knowledge this was not an option. I, for one, did not appreciate the finality of my input that night.

Site Clarification

Two respondents raised concerns about the perception of survey sites as being targeted for acquisition. While it was repeatedly stated the sites simply have the potential for openspace protection, and that sites are not under formal consideration, some residents obviously did not understand that focusing on specific important places is a necessary part of prioritizing land in a given area. Therefore, the Planning Commission should be aware of this concern as it engages further with the public on openspace planning issues.

• By asking survey participants to rate specific properties, it could be misinterpreted that those parcels were being considered for purchase by the town...greater care should be taken to avoid this giving impression.
• Need to make it more clear that these are not actual sites under consideration.

Sample Size

Three respondents raised concerns about the number of people who took the survey. As discussed in the survey findings section, this is qualitative research. While a larger sample size would have facilitated further and more complex analyses, it is not necessary that the majority of Charlestown's 7,859 residents take the survey before solid conclusions and findings be made. The representativeness of participants with the general Charlestown population allowed for strong conclusions to be drawn from this study. One respondent raised a concern about self-selection of survey participants. This survey was not conducted by employing a strict random sample-selection. All survey techniques are subject to some degree of self-selection. Whether sample bias is introduced in the solicitation process or by participants' willingness to extend interest to the survey, it is important to keep limitations in mind and focus on how representative respondents are with the larger Charlestown population. Because people may not understand the methodology of qualitative research, it is important that the Planning Commission articulates this process with concerned citizens.

• The number of survey participants was too small...more effective public notice and/or other methods for collecting survey information should be considered.
• My biggest concern is how to get more respondents so the results of the survey reflect a larger percentage of the towns citizens. I think the survey gathers a lot of information but the real power of that info for solid planning purposes is in the number of respondents. Since I sit on the Planning Commission I believe we have some responsibility to try to get more input.
• I would be very concerned about the representativeness of 124 respondents to this web survey. While the web provided some visual options that enhance the survey, I detected no method by which you controlled for sample-selection biases. For example, did you develop some sort of random sampling contact-list to encourage randomly chosen individuals to get on the web? How can you control for the difference between enthusiastic respondents and those who are less involved but still may pay or absorb some of the opportunity costs of having the town follow the conservation plan? Seems like these types of weaknesses are similar to those weaknesses associated with the public workshop process.

Insufficient Publicity

Three respondents raised concern about insufficient publicity for this study. As mentioned in the limitations section, publicity for this project could have been pursued more actively by local officials. This study could have been publicized in Charlestown's monthly newsletter. Furthermore, greater publicity could have been created by employing Charlestown's official website, and increasing efforts to poster the community.

• Finding out about it was purely by chance in the Westerly Sun. Some who took it have time to run around town and chit chat - some day I shall too, but I feel communication might be better addressed.
• 124 participants is fairly pitifull..maybe it should have been advertised that an individual would be available to assist people with the survey at the library. Not everyone has access to a computer.
• Since it would be extremely beneficial to increase the Sample Size, you should use every means at your disposal to obtain Resident's e-mail addresses.

Time

Despite efforts to decrease necessary time required to comment on sites by randomizing the survey, two respondents raised concern about survey length. This may be attributed to slower connections to the Internet. However, there is a tradeoff that must be made between the amount of information that is sought for collection and the amount of time asked of participants. The follow-up survey found that most participants took approximately 20 minutes to complete the survey. As stated in the Instructions, the survey was intended to take 15-20 minutes. Therefore, it is reasonable to conclude that a survey of this length is reasonable for residents.

• It took me over an hour. Since I need to conserve time on the Internet (one phone line) I would have liked to know ahead of time what I would be doing. You may have done this in a general way, but it didn't sink in! I'm curious as to how you are going to use such information.
• Needs to be shorter.

Survey as Awareness Raising Tool and
Means of Expanding Opportunities for Engaging Residents

One respondent made a comment that points to the survey's function as a means of raising awareness. After taking the survey, this resident was drawn to attend a meeting concerning one of the sites included in this study. By engaging residents with an Internet survey, the Planning Commission increases its chances for public participation in not only prioritizing land for openspace acquisition, but also in planning issues that are related to sites throughout Charlestown. The Planning Commission should make efforts to sustain public engagement in openspace and other land management planning.

• I did notice that, a few days after taking the survey, the United Nuclear property was designated as protected openspace. It was my lowest priority for this kind of use. Without the survey, I would not have attended a meeting to respond. Even if I had attended [the workshops], you would not have gotten the level and detail of response. I think this is a terrific way to get feedback--if you can get people to participate.

Comment on the Method

Four respondents commented on the web based method of presenting the survey. One respondent suggested an alternative approach for presenting survey sites, and another suggested an improvement for visual representations of the sites. These comments show that residents are not only willing and able to give valuable input for survey questions, but they are also thinking of ways to improve the method for collecting public input in the prioritization process. Further prioritization efforts should consider such comments for improving the data gathering process.

• The best thing about the survey for me is recognizing that there is some effort and concern regarding the protection and maintenance of Charlestown. Perhaps an overview of Charlestown with a clickable link for people to direct there particular area of concern to, followed with a dialog box like this one to voice the concerns. All in all I see it as being a useful tool for gathering initial data and response to the survey. Thanks for letting me contribute
• I couldn't recognize one of the sites from the photo but should have because it is close to our residence. Cross Mills was a reference point. A better map would have been helpful.
• I liked the web survey approach a great deal. I thought it covered the issues accurately.
• I think it was just fine as it was.

Follow-up Survey Results