<!DOCTYPE html>

<html data-programme="Research Integrity" data-course="Managing and Recording Your Research">

	<head>
		<title>Validation</title>
		<meta charset="UTF-8">
		<meta http-equiv="X-UA-Compatible" content="IE=edge">
		<meta name="viewport" content="width=device-width, initial-scale=1">

		<link rel="stylesheet" type="text/css" href="../css/project.min.css">
		<link id="basic-print" rel="stylesheet" type="text/css" href="../css/print.min.css" disabled>
		<link id="summary-print" rel="stylesheet" type="text/css" href="../css/print_interactive.min.css">
	<script type="text/javascript" src="../scripts_scorm_12/SCORM_12_APIWrapper.js"></script>
<script type="text/javascript" src="../scripts_scorm_12/courseFunctions.js"></script>
</head>

	<body onLoad="javascript:Initialize();" onUnload="javascript:Terminate();">
	  
		<div id="app" v-cloak>
			<base-container
				screen-id="cre_1_50"
				current-module = "Managing and Recording Your Research"
				page-title = "Validation"
				screen-time = "5"
				screen-time-period = "mins">

				<page-container>

					<page-row>
						<page-content-main>
							
							<p>If someone challenges your findings, you must be able to provide evidence that you discovered what you did in the way you described; that is, you must be able to validate your findings.</p>
							<p>For some types of research, validation procedures can be simple, although they are not always followed. Validation can be more difficult for other types of research.</p>
							
							<h3>Validation</h3>
							
							<p>Many research areas have well-established validation procedures. These range from the use of mandatory validation software in fields such as x-ray crystallography, through to qualitative procedures such as <glossary content="Showing transcripts to interviewees/focus group participants to check that they are accurate and convey the intended meaning.">member checking</glossary>. <glossary content="Evidence of the time and date when the digital information contained in your research record was created and last amended. This evidence is stored by an independent provider to prevent it from being tampered with.">Independent digital time stamps</glossary> can be very helpful to researchers from various disciplines who are seeking to validate their recorded data.</p>
								
							<instructional-text><p>The activity below sets out some common validation practices. Select the response that matches your evaluation of the reliability of the method described, and click "Check Answer" to access our thoughts. You should then click "Next" to evaluate the next method.</p></instructional-text>
							<instructional-text-accessible><p>The next activity sets out some common validation practices. Select the response that matches your evaluation of the reliability of the method described and consider our feedback. Then continue on to evaluate the next method.</p></instructional-text-accessible>

							<!-- ACTIVITY START -->
							<process restart="true" variant="mcq" show-in-accessible-version="false">
								<process-page>
									<mcq>
										<mcq-question><p>Have the record reviewed, dated and signed (written or digitally) by someone else on a regular basis.</p></mcq-question>
										<mcq-options>
											<mcq-option neutral>
												<mcq-answer>Definitely reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely reliable</h4>
                                					<p>Even if this practice is followed, the research record could still be unreliable (for example, if the record is written in pencil or if it does not have an independent digital time stamp).</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>It depends...</mcq-answer>
												<mcq-feedback>
													<h4>It depends...</h4>
                                					<p>The reliability of this method depends on whether the record could be changed after it was signed without anyone knowing it had been changed. The key to reliability is creating an unalterable, datable record.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>Definitely not reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely not reliable</h4>
                                					<p>This practice could be reliable if the research record is written in ink on numbered pages, or if it has been given an independent digital time stamp.</p>
												</mcq-feedback>
											</mcq-option>
										</mcq-options>
									</mcq>
								</process-page>
								<process-page>
									<mcq>
										<mcq-question><p>Photocopy the record and store it in a safe place.</p></mcq-question>
										<mcq-options>
											<mcq-option neutral>
												<mcq-answer>Definitely reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely reliable</h4>
                                					<p>Since the photocopy could have been made at any time, this is not a reliable method for validating your research record. An untrustworthy researcher could change what was recorded after the project was finished, photocopy the altered results, and file them away as a 'true' record of their findings.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>It depends...</mcq-answer>
												<mcq-feedback>
													<h4>It depends...</h4>
                                					<p>For this method to be reliable, you must find a way to document when the photocopy was made. There are ways to do this, such as sending it to yourself in a sealed envelope that you leave unopened &ndash; however, such methods are generally impractical. Rather than a paper photocopy, a digital copy with an independent digital time stamp would more effectively ensure the validity of the research record.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>Definitely not reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely not reliable</h4>
                                					<p>Since it is difficult to identify when a photocopy was made, this method is not reliable. A digital copy with an independent time stamp would more effectively ensure the validity of the research record.</p>
												</mcq-feedback>
											</mcq-option>
										</mcq-options>
									</mcq>
								</process-page>
								<process-page>
									<mcq>
										<mcq-question><p>Rely on the date and time assigned by your PC to all the documents that form the research record.</p></mcq-question>
										<mcq-options>
											<mcq-option neutral>
												<mcq-answer>Definitely reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely reliable</h4>
                                					<p>This is usually not a reliable method. The information associated with a digital file can be altered, often with ease.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>It depends...</mcq-answer>
												<mcq-feedback>
													<h4>It depends...</h4>
                                					<p>This is generally not a reliable method, since the information associated with a digital file can be altered. If you do not control and cannot change the date&ndash;time system in the computer you are using, then this could be a reliable method.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>Definitely not reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely not reliable</h4>
                                					<p>This would only be a reliable method if you do not control and cannot change the date&ndash;time attribution system of the computer you are using. Otherwise, this is generally not a reliable method, since the information associated with a digital file can be altered by changing the date and time assigned to the files.</p>
												</mcq-feedback>
											</mcq-option>
										</mcq-options>
									</mcq>
								</process-page>
								<process-page>
									<mcq>
										<mcq-question><p>Maintain a bound copy of the research record signed at regular intervals by a supervisor or colleague.</p></mcq-question>
										<mcq-options>
											<mcq-option neutral>
												<mcq-answer>Definitely reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely reliable</h4>
                                					<p>This method was once considered one of the more reliable means of validating a research record &ndash; binding makes a record difficult to tamper with, and getting the signature of a supervisor or colleague periodically indicates that the record has received oversight from an external party. The one caution is that the information must be recorded in such a way that it cannot be changed, which usually means it must be written in ink on numbered pages, with any white space crossed through. In many fields, bound notebooks have been largely replaced by digital notebooks for both convenience and storage capacity.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>It depends...</mcq-answer>
												<mcq-feedback>
													<h4>It depends...</h4>
                                					<p>You are right; there are reasons for caution. Keeping a bound copy and ensuring it receives oversight from others is usually a reliable way to validate your work, but care must be taken to ensure the information is recorded in a way that cannot be changed.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>Definitely not reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely not reliable</h4>
                                					<p>While there is more to validating a research record than this, keeping a bound copy and ensuring it receives oversight from others is generally accepted as a reliable way to validate your work, provided the information is recorded in a way that cannot be changed.</p>
												</mcq-feedback>
											</mcq-option>
										</mcq-options>
									</mcq>
								</process-page>
								<process-page>
									<mcq>
										<mcq-question><p>Back up your digital notebook or data files once a day.</p></mcq-question>
										<mcq-options>
											<mcq-option neutral>
												<mcq-answer>Definitely reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely reliable</h4>
                                					<p>This method is appropriate as long as your backup system has a way to verify the date of the backup. If the backup system just copies the time and date your computer assigns to your files, the backup date will change if you adjust the date on your computer. For this approach to be valid, the dating system used for the backup must be independent of your computer.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>It depends...</mcq-answer>
												<mcq-feedback>
													<h4>It depends...</h4>
                                					<p>You are right; there are reasons for caution. If the backup system just copies the time and date your computer assigns to your files, the backup date will change if you change the date on your computer. For this approach to be valid, the dating system used for the backup must be independent of your computer.</p>
												</mcq-feedback>
											</mcq-option>
											<mcq-option neutral>
												<mcq-answer>Definitely not reliable</mcq-answer>
												<mcq-feedback>
													<h4>Definitely not reliable</h4>
                                					<p>This method is not appropriate if your backup system cannot verify the date of the backup. If the backup system just copies the time and date your computer assigns to your files, the backup date will change if you change the date on your computer. For this approach to be valid, the dating system used for the backup must be independent of your computer.</p>
												</mcq-feedback>
											</mcq-option>
										</mcq-options>
									</mcq>
								</process-page>
							</process>
							<!-- ACCESSIBLE START -->
							<accessible-block>
								<h4>Choose a category for each statement from this list:</h4>
								<ul>
									<li>Definitely reliable</li>
									<li>It depends...</li>
									<li>Definitely not reliable</li>
								</ul>
								<hr>
								<h4>Statement 1 of 5:</h4>
								<p>Have the record reviewed, dated and signed (written or digitally) by someone else on a regular basis.</p>
								<p><strong>Feedback for "Definitely reliable":</strong> Even if this practice is followed, the research record could still be unreliable (for example, if the record is written in pencil or if it does not have an independent digital time stamp).</p>
								<p><strong>Feedback for "It depends...":</strong> The reliability of this method depends on whether the record could be changed after it was signed without anyone knowing it had been changed. The key to reliability is creating an unalterable, datable record. </p>
								<p><strong>Feedback for "Definitely unreliable":</strong> This practice could be reliable if the research record is written in ink on numbered pages, or if it has been given an independent digital time stamp.</p>
								<hr>
								<h4>Statement 2 of 5:</h4>
								<p>Photocopy the record and store it in a safe place.</p>
								<p><strong>Feedback for "Definitely reliable":</strong> Since the photocopy could have been made at any time, this is not a reliable method for validating your research record. An untrustworthy researcher could change what was recorded after the project was finished, photocopy the altered results, and file them away as a 'true' record of their findings.</p>
								<p><strong>Feedback for "It depends...":</strong> For this method to be reliable, you must find a way to document when the photocopy was made. There are ways to do this, such as sending it to yourself in a sealed envelope that you leave unopened &ndash; however, such methods are generally impractical. Rather than a paper photocopy, a digital copy with an independent digital time stamp would more effectively ensure the validity of the research record.</p>
								<p><strong>Feedback for "Definitely unreliable":</strong> Since it is difficult to identify when a photocopy was made, this method is not reliable. A digital copy with an independent time stamp would more effectively ensure the validity of the research record.</p>
								<hr>
								<h4>Statement 3 of 5:</h4>
								<p>Rely on the date and time assigned by your PC to all the documents that form the research record. </p>
								<p><strong>Feedback for "Definitely reliable":</strong> This is usually not a reliable method. The information associated with a digital file can be altered, often with ease.</p>
								<p><strong>Feedback for "It depends...":</strong> This is generally not a reliable method, since the information associated with a digital file can be altered. If you do not control and cannot change the date&ndash;time system in the computer you are using, then this could be a reliable method.</p>
								<p><strong>Feedback for "Definitely unreliable":</strong> This would only be a reliable method if you do not control and cannot change the date&ndash;time attribution system of the computer you are using. Otherwise, this is generally not a reliable method since the information associated with a digital file can be altered by changing the date and time assigned to the files.</p>
								<hr>
								<h4>Statement 4 of 5:</h4>
								<p>Maintain a bound copy of the research record signed at regular intervals by a supervisor or colleague.</p>
								<p><strong>Feedback for "Definitely reliable":</strong> This method was once considered one of the more reliable means of validating a research record &ndash; binding makes a record difficult to tamper with, and getting the signature of a supervisor or colleague periodically indicates that the record has received oversight from an external party. The one caution is that the information must be recorded in such a way that it cannot be changed, which usually means it must be written in ink on numbered pages, with any white space crossed through. In many fields, bound notebooks have been largely replaced by digital notebooks for both convenience and storage capacity.</p>
								<p><strong>Feedback for "It depends...":</strong> You are right; there are reasons for caution. Keeping a bound copy and ensuring it receives oversight from others is usually a reliable way to validate your work, but care must be taken to ensure the information is recorded in a way that cannot be changed.</p>
								<p><strong>Feedback for "Definitely unreliable":</strong> While there is more to validating a research record than this, keeping a bound copy and ensuring it receives oversight from others is generally accepted as a reliable way to validate your work, provided the information is recorded in a way that cannot be changed.</p>
								<hr>
								<h4>Statement 5 of 5:</h4>
								<p>Back up your digital notebook or data files once a day.</p>
								<p><strong>Feedback for "Definitely reliable":</strong> This method is appropriate as long as your backup system has a way to verify the date of the backup. If the backup system just copies the time and date your computer assigns to your files, the backup date will change if you adjust the date on your computer. For this approach to be valid, the dating system used for the backup must be independent of your computer.</p>
								<p><strong>Feedback for "It depends...":</strong> You are right; there are reasons for caution. If the backup system just copies the time and date your computer assigns to your files, the backup date will change if you change the date on your computer. For this approach to be valid, the dating system used for the backup must be independent of your computer.</p>
								<p><strong>Feedback for "Definitely unreliable":</strong> This method is not appropriate if your backup system cannot verify the date of the backup. If the backup system just copies the time and date your computer assigns to your files, the backup date will change if you change the date on your computer. For this approach to be valid, the dating system used for the backup must be independent of your computer.</p>
							</accessible-block>
							<!-- ACCESSIBLE END -->
							<!-- ACTIVITY END -->
							
							<p>As you plan and follow through on your approach to validation, keep in mind that any approach you use must be consistent with any requirements for data protection. Records that contain personal data must be kept and validated in a secure environment. It will also be important to check the conditions at and rules set by your institution &ndash; for example, some institutions may frown on the use of services that offer independent digital time stamping, since these may represent a data security risk. Your institution should be able to refer you to services or tools that meet its requirements.</p>
							
						</page-content-main>
					</page-row>

					<page-row>
						<page-content-main>

							<h3>Validation challenges</h3>
							
							<p>Validation can be a challenge in some fields of research. As projects become more complex, it is difficult to record everything that could be related to the results you are observing. In addition, researchers are not always aware of the factors that influence what they record, and what they record may not itself be valid (see the "Additional information" pod for further discussion of this).</p>

						</page-content-main>

						<page-content-sidebar>

							<!-- ADDITIONAL INFORMATION START -->
							<pod variant="b" has-modal="true">
								<pod-heading>Additional information</pod-heading> <!-- pod title -->
								<pod-content></pod-content>
								<!-- pop-up pod content -->
								<pod-modal-heading>Additional information</pod-modal-heading>
								<pod-modal-content>
									<hosted-link id="RI_INT_cre_1_50_1">
										<p>Thinking carefully about record-keeping is essential no matter what type of research you are conducting.</p>
										<p>For instance, if you are conducting online research you should be aware that websites are frequently edited or even taken down. This can make it difficult to prove what you were referring to further down the line, which can impede validation.</p>
										<p>For an example of the consequences of this, see this <a href="https://onlinelibrary.wiley.com/doi/full/10.1002/pds.3308" target="_blank">retraction notice</a>.</p>
									</hosted-link>
								</pod-modal-content>
							</pod>
							<!-- ADDITIONAL INFORMATION END -->

						</page-content-sidebar>
					</page-row>

					<page-row>
						<page-content-main>

							<instructional-text class="non-interactive-swap-out"><p>Choose a headline from the options below to read about a case that demonstrates the need for rigorous validation procedures. Reflect on the scenario in relation to the questions posed, enter your thoughts, and then click "View feedback" to get our take. To reflect on another scenario, return to the "Main menu" and choose a different headline.</p></instructional-text>

						</page-content-main>
					</page-row>

					<page-row>
						<page-content-main>

							<instructional-text-accessible class="non-interactive-swap-in"><p>Choose a headline from the list to learn about a case that demonstrates the need for rigorous validation procedures. Reflect on the scenario in relation to the questions posed, consider what your response would be and then proceed for our feedback. To reflect on another scenario, choose another headline from the list.</p></instructional-text-accessible>
							
							<!-- ACTIVITY START -->
							<action-maze restart-label="Main menu" back="false" summary="true" variant="notemaking" show-in-accessible-version="false" class="non-interactive-swap-out">
								<action-panel panel="0">
									<action-link goto-panel="1"><h3>False coding of observational studies</h3></action-link>
									<action-link goto-panel="2"><h3>Difficulty replicating economic studies</h3></action-link>
									<action-link goto-panel="3"><h3>Failure to confirm records concerning heart failure</h3></action-link>
								</action-panel>
								<action-panel panel="1">
									<note-making summary="false">
										<note-question>
											<p>In 2012, the US Office of Research Integrity found Harvard University psychologist Marc Hauser guilty of six counts of research misconduct. Among other offences, he was found guilty of falsely coding the results of observational studies of animal behaviour.</p>
											<p>A University of Maryland physicist wasn't so sure that the behaviour was deliberate, suggesting "It's entirely possible that Hauser was swayed by 'confirmation bias' &ndash; the tendency to look for and perceive evidence consistent with our hypotheses and to deny, dismiss or distort evidence that is not" (Lilienfeld, 2010).</p>
											<p><strong>In this case, there was ambiguity over the question of whether Hauser had recorded what he thought had happened rather than what had actually occurred. How could Hauser have validated what had happened in his research?</strong></p>
										</note-question>
										<note-input></note-input>
										<note-feedback><p>In this case, it would have been appropriate to have two or more researchers view and code the same event. If the coding is the same across different researchers, the chance of bias is reduced. If there are differences, they can be reviewed for possible bias or other factors that could influence the observations.</p></note-feedback>
									</note-making>
								</action-panel>
								<action-panel panel="2">
									<note-making summary="false">
										<note-question>
											<p>A study published in 2016 reported that the results in 7 of 18 (39%) economic studies could not be replicated using the same predefined analysis plans (Bohannon, 2016).  The lack of replicability (also documented in psychology and other fields) raises questions about the reliability of the research.</p>
											<p><strong>How much information should researchers include in their research record to ensure that problems such as this do not occur? How do you think this information can be validated?</strong></p>
										</note-question>
										<note-input></note-input>
										<note-feedback><p>In some fields, such as economics, it is often difficult to replicate studies since factors can change very quickly, making replication more difficult.  The best advice is to be as comprehensive as possible when explaining how you conducted your study. To validate your own work, make sure you are able to produce the evidence you used for drawing conclusions.</p></note-feedback>
									</note-making>
								</action-panel>
								<action-panel panel="3">
									<note-making summary="false">
										<note-question>
											<p>A 2017 study sought to analyse the proportion of registered heart failure diagnoses that could be confirmed by referring to primary care medical records, as well as analysing related factors. Over the course of the study, the researchers found that only half of the cases labelled as heart failure in primary care medical records could have their diagnosis confirmed by further analysis (Verdú-Rotellar <em>et al.</em>, 2017).</p>
											<p><strong>What challenges for validation does this study highlight? How does validation become more complex when researchers have to rely on the methods of others?</strong></p>
										</note-question>
										<note-input></note-input>
										<note-feedback><p>During the course of your research, you may encounter flawed research or incomplete records in databases or elsewhere. This may mean that you have to make adjustments if you have doubts about the validity or comprehensiveness of your intended dataset. Your method could be flawless, but flawless analysis will not turn flawed information into reliable information.</p></note-feedback>
									</note-making>
								</action-panel>
							</action-maze>
							<!-- ACCESSIBLE START -->
							<accessible-block class="non-interactive-swap-in">
								<h4>Headlines:</h4>
								<ul>
									<li><strong>False coding of observational studies</strong> [Proceed to Question 1]</li>
									<li><strong>Difficulty replicating economic studies</strong> [Proceed to Question 2]</li>
									<li><strong>Failure to confirm records concerning heart failure</strong> [Proceed to Question 3]</li>
								</ul>
								<hr>
								<h4>Headline 1 of 3: False coding of observational studies</h4>
								<p><strong>Question 1:</strong> In 2012, the US Office of Research Integrity found Harvard University psychologist Marc Hauser guilty of six counts of research misconduct. Among other offences, he was found guilty of falsely coding the results of observational studies of animal behaviour.</p>
								<p>A University of Maryland physicist wasn't so sure that the behaviour was deliberate, suggesting "It's entirely possible that Hauser was swayed by 'confirmation bias' &ndash; the tendency to look for and perceive evidence consistent with our hypotheses and to deny, dismiss or distort evidence that is not" (Lilienfeld, 2010).  </p>
								<p><strong>In this case, there was ambiguity over the question of whether Hauser had recorded what he thought had happened rather than what had actually occurred. How could Hauser have validated what had happened in his research?</strong></p>
								<p><strong>Feedback:</strong> In this case, it would have been appropriate to have two or more researchers view and code the same event. If the coding is the same across different researchers, the chance of bias is reduced. If there are differences, they can be reviewed for possible bias or other factors that could influence the observations.</p>
								<hr>
								<h4>Headline 2 of 3:</strong> Difficulty replicating economic studies</h4>
								<p><strong>Question 2:</strong> A study published in 2016 reported that the results in 7 of 18 (39%) economic studies could not be replicated using the same predefined analysis plans (Bohannon, 2016). The lack of replicability (also documented in psychology and other fields) raises questions about the reliability of the research.</p>
								<p><strong>How much information should researchers include in their research record to ensure that problems such as this do not occur? How do you think this information can be validated?</strong></p>
								<p><strong>Feedback:</strong> In some fields, such as economics, it is often difficult to replicate studies since factors can change very quickly, making replication more difficult. The best advice is to be as comprehensive as possible when explaining how you conducted your study. To validate your own work, make sure you are able to produce the evidence you used for drawing conclusions.</p>
								<hr>
								<h4>Headline 3 of 3: Failure to confirm records concerning heart failure</h4>
								<p><strong>Question 3:</strong> A 2017 study sought to analyse the proportion of registered heart failure diagnoses that could be confirmed by referring to primary care medical records, as well as analysing related factors. Over the course of the study, the researchers found that only half of the cases labelled as heart failure in primary care medical records could have their diagnosis confirmed by further analysis (Verdú-Rotellar <em>et al.</em>, 2017).</p>
								<p><strong>What challenges for validation does this study highlight? How does validation become more complex when researchers have to rely on the methods of others?</strong></p>
								<p><strong>Feedback:</strong> During the course of your research, you may encounter flawed research or incomplete records in databases or elsewhere. This may mean that you have to make adjustments if you have doubts about the validity or comprehensiveness of your intended dataset. Your method could be flawless, but flawless analysis will not turn flawed information into reliable information.</p>
							</accessible-block>
							<!-- ACCESSIBLE END -->
							<!-- ACTIVITY END -->
							
							<h3>Summary</h3>
							
							<p>Properly validating your research record is essential. At any time you could be asked:</p>
							
							<ul>
								<li>"I tried but could not replicate your work. Can you provide me with an exact account of what you did?"</li>
								<li>"A colleague has suggested that some of your results may have been fabricated after the work was completed.  Can you provide me with a copy of the raw data and evidence of when it was collected?"</li>
								<li>"We would like to give you credit for this discovery but a colleague at another university claims to be the source of your findings. Do you have validated records demonstrating how your work in this area evolved over time?"</li>
							</ul>
							
							<p>Properly validating your research puts you in the best position to answer questions such as these with confidence. The time and effort you put into validation may go unnoticed until you have to provide evidence of what you did, which is when you will find the energy you invested in the validation process invaluable.</p>
							
						</page-content-main>
						
						<page-content-sidebar>
							
							<!-- USEFUL LINKS START -->
							<pod variant="a" has-modal="true">
								<pod-heading>Useful links</pod-heading> <!-- pod title -->
								<pod-content></pod-content>
								<!-- pop-up pod content -->
								<pod-modal-heading>Useful links</pod-modal-heading>
								<pod-modal-content>
									<hosted-link id="RI_INT_cre_1_50_2">
										<ul>
											<li><a href="https://www.scientificamerican.com/article/fudge-factor/" target="_blank">Fudge Factor: A Look at a Harvard Science Fraud Case</a>, <em>Scientific American</em></li>
											<li><a href="http://www.sciencemag.org/news/2016/03/about-40-economics-experiments-fail-replication-survey" target="_blank">About 40% of Economics Experiments Fail Replication Survey</a>, <em>Science</em></li>
											<li><a href="https://www.tandfonline.com/doi/full/10.1080/13814788.2017.1305104" target="_blank">Validation of Heart Failure Diagnosis Registered in Primary Care Records in Two Primary Care Centres in Barcelona (Spain) and Factors Related. A Cross-sectional Study</a>, <em>European Journal of General Practice</em>.</li>
										</ul>
									</hosted-link>
								</pod-modal-content>
							</pod>
							<!-- USEFUL LINKS END -->

							<interactive-text>
								<!-- DISCIPLINE-SPECIFIC ICON START -->
								<icon-discipline-specific></icon-discipline-specific>
								<!-- DISCIPLINE-SPECIFIC ICON END -->
							</interactive-text>
							
						</page-content-sidebar>
					</page-row>
					
				</page-container>
			</base-container>
		</div> <!-- END content wrapper -->
		
		<!-- scripts -->
		<script src="../js/vendor.min.js"></script>
		<script src="../js/vue-components.min.js"></script>
		<script>
			new Vue({
			  el: "#app",
			  data: {},
			  mounted: function () {
				var core = document.createElement('script');
				core.src = "../js/core.min.js";
				document.getElementsByTagName('head')[0].appendChild(core);

				var project = document.createElement('script');
				project.src = "../js/project.min.js";
				document.getElementsByTagName('head')[0].appendChild(project);
			  }
			});
		</script>

	</body>
</html>