«Abstract Volunteered geographic information (VGI) is a potential source of information to complement other sources in ﬂood risk management. ...»
The average of volunteered information was calculated by the sum of the values observed by each participant at one point and divided by the number of observations. It is noteworthy that, in this context, a volunteer information refers to the value of the water level in the water level ruler. This is an interpretation mechanism laid down on the riverbed that allows a volunteer to determine the water level. On the other hand, the average of sensor data was calculated by the sum of measurements carried out by a sensor during the period of time that the participants performed observations, and divided by the number of measurements.
To analyze the results we used four statistical tests: Shapiro-Wilk Test, Levene’s Test, T Test and Mann-Whitney Test. The Shapiro-Wilk Test  was used http://www.agora.icmc.usp.br/enchente to check whether the samples had normal distribution. If samples had normal distribution, the Levene’s Test  was applied for verifying if the samples had equal variance. Then the T Test  was performed to statistically compare the sample’s average and thus reject or accept the null hypothesis. If samples did not have a normal distribution, the Mann-Whitney Test  was performed.
3.1 Instumentation The main diﬀerence between the ﬁrst experiment and this evaluation is related to the number and knowledge of participants, and the number of collection points.
Unlike the ﬁrst experiment, which included the participation of 10 volunteers, 15 new volunteers participated of this experimental evaluation. Thus, the increase in the number of participants improves the conﬁdence in results.
In the ﬁrst experiment, 50% of participants had at least one month of experience with ﬂood risk management. In this experimental evaluation, the participants selected had no experience with this context. The lack of experience of participants approaches the experiment to the real scenario, since most of the aﬀected population have no experience with the context of ﬂood risk management. Furthermore, the crowdsourcing-based approach, proposed by Degrossi et al. , aims at providing mechanisms to help the population in providing useful information for ﬂood risk management context.
Besides the changes related to participants, in this experimental evaluation, information gathering was carried out in two distinct points of the watershed of S˜o Carlos/SP city. It is worth noting that both points are equipped with a a water level ruler, that it is used as a mechanism to help participants in better interpreting environmental variable, i.e. water level.
Before the experiment, participants went through a training. The training addressed three points: (i ) the mechanisms used to interpret environmental variables; (ii ) the crowdsourcing platform used to obtain volunteered information;
and (iii ) instructions about how to insert a report in the platform.
Degrossi et al.  proposed four interpretation mechanism for helping volunteers in better interpreting environmental variables, i.e. water level. The ﬁrst mechanism is a water level ruler that it is laid down in the riverbed. In this experiment, only this mechanism was used by volunteers to interpret the water level in the riverbed. This mechanism was chosen because it oﬀers a higher accuracy of the water level measure, so it is possible to make a comparison between the measurements carried out by sensors and the observations made by volunteers.
The second mechanism proposed is a dummy with the human form, which has marks that represent the water level. The third mechanism corresponds to a multi-color bands, which represents the hazard index . This index represents the level of danger to the population, this is, the forces exerted on an individual by the water. Finally, the fourth mechanism may be used in points that do not have any of the mechanisms mentioned earlier. This mechanism is based on the popular knowledge, this is, water level is determined based on four marks: low, normal, high and overﬂowing.
The interpretation mechanisms are used as part of a report inserted in the crowdsourcing platform, which was used for gathering volunteered information.
The crowdsourcing platform aims at obtaining useful volunteered information for ﬂood risk management. When inserting a report, a volunteer must provide four mandatory information: (i ) report title; (ii ) report description; (iii ) category, which represents the interpretation mechanism used in the observation; and (iv ) the name of the local.
The conduction of the experiment was carried out in three steps. In the ﬁrst step, participants collected information about the water level in the water level ruler at the ﬁrst point, called Point A. In the second step, participants also collected information about the water level in the water level ruler, but at the second point, called Point B. Finally, in the third step, participants inserted their observations in the crowdsourcing platform.
It is noteworthy that during the three steps, participants were instructed to not exchange information with each other, avoiding bias in the information gathering. In addition, all steps were followed by the researchers responsible for the experiment.
4 Results and Discussion The experiment was conducted with 15 participants, with no prior knowledge about ﬂood risk management, in two points of the watershed of the city of S˜o a Carlos/SP. The results of this experimental evaluation were based on the four statistical tests, as mentioned in Section 3.
Information inserted by participants in the crowdsourcing platform were used to verify if the diﬀerence between the average of volunteer information and the average of sensor data is statistically signiﬁcant. It is worth mentioning that each point was considered independently for the statistical analysis. In the following sections we present the results for each point.
4.1 Point A For the statistical analysis related to Point A, 15 volunteered information, inserted by participants in the crowdsourcing platform, and 11 sensor data, which were measured during the experiment, were used (Table 1).
The ﬁrst statistical test carried out was Shapiro-Wilk Test, in order to determine whether the samples had a normal distribution. In particular, the signiﬁcance level3 adopted for this test was 5%. The results (Table 1) show that both samples have normal distribution, since the value of Sig. (p-value) is higher than the signiﬁcance level for both samples.
Table 1. Shapiro-Wilk Test Results (Point A).
According to Shapiro-Wilk Test results, T Test was carried out for verifying if the diﬀerence between the average of each sample is statistically signiﬁcant. Before this test, it is necessary to analyze whether the samples have equal variance.
The signiﬁcance level corresponds the probability to reject the null hypothesis being that true.
Therefore, we performed Levene’s Test. The results (Table 2) show that the samples do not have equal variance, since the value of Sig. (p-value) is smaller than the signiﬁcance level. Thus, at the T Test, we observed the results for samples that do not have equal variance.
Finally, T Test was carried out in order to reject the null hypothesis. The result (Table 2) shows that the null hypothesis can not be refuted, since the value of Sig. (p-value) is higher than the signiﬁcance level. Te can conclude that there is not statistically signiﬁcant diﬀerence between the average of volunteer information and the average of sensor data, i.e. the average are equal with a conﬁdence level of 95%.
The results indicate that volunteered information is useful for ﬂood risk management context, even with the lack of experience of participants. This can be explained by the fact that participants went through a training before the execution. Thus, we can assert that the training was suﬃcient for enabling participants to produce useful volunteered information for ﬂood risk management.
It is noteworthy that the water level ruler, used in information gathering, had a slight deviation on its bottom, which can be considered a threat to validity of the study. However, since this is a real scenario, this threat approaches the experiment to the real use proposed in the approach.
Table 2. Results of Levene’s Test and T Test (Point A).
Levene’s Test T Test Tested Variable Sig. Sig.
Water Level 0,001 0,750
4.2 Point B For the statistical analysis related to Point B, 13 volunteer information was provided by participants, while 21 sensor data were measured during the conduction of the experiment (Table 3).
Similar to Point A, the ﬁrst statistical test performed was Shapiro-Wilk Test, in order to verify if the samples had normal distribution. The signiﬁcance level adopted for this test was 5%. However, unlike point A, the results (Table 3) indicated that both samples do not have normal distribution, since the value of Sig. (p-value) is smaller than the signiﬁcance level for both samples. Thus, for samples that do not have normal distribution, the most appropriate hypothesis test is the Mann-Whitney Test.
Table 3. Shapiro-Wilk Test Results (Point B).
The Mann-Whitney Test, as well as T Test, was carried out for verifying if the diﬀerence between the average of samples from Point B is statistically signiﬁcant.
The results (Table 4) show that the null hypothesis can not be rejected, as well as at Point A, since the value of Sig. (p-value) is higher than the signiﬁcance level, i.e. the average of both samples at Point B are equal with a conﬁdence level of 95%. Thus, we can assert that volunteer information is useful for context of ﬂood risk management.
Table 4. Results of Mann-Whitney Test (Point B).
Mann-Whitney Test Tested Variable Sig.
Water Level 0,059 As at Point A, the water level ruler also had factors that may threaten the validity of the study. However, such factors approximate the experiment of the real conditions found by volunteers. In the case of Point B, the bottom of the water level ruler was obstructed by vegetation, as shown in Figure 1. This obstruction hindered the determination of the water level by the volunteer, which could cause erroneous values.
Figure 1. Water level ruler at Point B.
5 Conclusion In this work, an experimental evaluation was carried out for verifying whether volunteered information can be useful for the ﬂood risk management context.
In the experiment, volunteers provided information about the water level in the riverbed using a crowdsourcing platform. The results showed that, with the use of interpretation mechanisms and training, volunteers were able to provide information about environmental variable (i.e. water levels) in an accurate way, i.e. they were able to provide useful information to the context of ﬂood risk management. We can make such assertion since the average of observations made by volunteered was statistically equivalent to the average of sensor data.
Furthermore, the use of the Flood Citizen Observatory, as a crowdsourcing platform, was able to mitigate the variability in the structure of volunteered observations, since the platform provides some standards for information provision.
Thus, recurring problems, such as the low or lack of structure of volunteered information, can be attenuated. The structure of volunteered information is the ﬁrst step towards integrating this type of information with data from other sources, such as sensors, rain gauges, among others.
Although the experiment was eﬀective for answering the research question, improvements should be carried out in this context. Volunteers had problems with the mobile application interface, i.e. some volunteers had diﬃculty in interpreting some report ﬁelds at the time to submit an observation. Another challenge of the usefulness of volunteered information is related to VGI quality. In particular, in the crowdsourcing platform, the quality assessment is carried out manually.
So, it is necessary to develop an automatic method for the evaluation of the quality of volunteered information.
Acknowledgment We would like to express our thanks for the ﬁnancial support provided by CAPES (Coordination for the Improvement of Higher Education Personnel, Brazil). Jo˜oa Porto de Albuquerque is grateful for FAPESP (grant no. FAPESP 2012/18675CAPES (grant no. 12065-13-7) and Heidelberg University (Excellence Initiative II / Action 7) for providing funding for his contribution to this paper.
1. UN, U.N.: Humanitarian and disaster relief assistance. Acessed on: 12/02/2013 (2013)
2. Alcantara-Ayala, I.: Geomorphology, natural hazards, vulnerability and prevention of natural disasters in developing countries. Geomorphology 47 (2002) 107–124
3. Levy, J.K., Gopalakrishnan, C., Lin, Z.: Advances in decision support systems for ﬂood disaster management: Challenges and opportunities. International Journal of Water Resources Development 21(4) (2005) 593–612
4. Poser, K., Dransch, D.: Volunteered geographic information for disaster management with application to rapid ﬂood damage estimation. Geomatica 64(1) (2010) 89–98
5. Tu, S., Abdelguerf, M.: Web services for geographic information systems. IEEE Internet Computing (2006) 13–15
6. Plate, E.J.: Flood risk and ﬂood management. Journal of Hydrology 267 (2002) 2–11
7. Kobiyama, M., Mendon¸a, M., Moreno, D.A., de Oliveira Marcelino, I.P.V., c Marcelino, E.V., Gon¸alves, E.F., Brazetti, L.L.P., Goerl, R.F., Molleri, G.S.F., c de Moraes Rudorﬀ, F.: Preven¸ao de Desastres Naturais - Conceitos b´sicos. Orc˜ a ganic Trading (2006)
8. van der Kooij, M.: Flood monitoring and disaster management response. GEOconnexion International Magazine (Abril 2013) 26–28
9. Goodchild, M.F.: Citizens as sensors: the world of volunteered geography. GeoJournal 69(4) (Agosto 2007) 211–221
10. Longueville, B.D., Luraschi, G., Smits, P., Peedell, S., Groeve, T.D.: Citizens as sensors for natural hazards: A vgi intergration workﬂow. Geomatica 64 (2010) 41–59