Next Article in Journal
Acknowledgment to the Reviewers of Education Sciences in 2022
Next Article in Special Issue
Context-Based Support to Enhance Developers’ Learning of Software Security
Previous Article in Journal
Integration of Sustainable Criteria in the Development of a Proposal for an Online Postgraduate Program in the Projects Area
Previous Article in Special Issue
Impacts of Technology in Learning: Mobile Typing Applications for Writing and Accomplishing Academic Tasks among Arabic-Speaking Undergraduate Students
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Education Sciences
Volume 13
Issue 2
10.3390/educsci13020098
education-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Saturation in Qualitative Educational Technology Research

by
Wajeeh Daher
1,2
1
Department of Mathematics, Al-Qasemi Academic College of Education, Baqa-El-Gharbia 3010000, Israel
2
Faculty of Educational Sciences, An-Najah National University, Nablus P400, Palestine
Educ. Sci. 2023, 13(2), 98; https://doi.org/10.3390/educsci13020098
Submission received: 24 December 2022 / Revised: 12 January 2023 / Accepted: 12 January 2023 / Published: 17 January 2023
(This article belongs to the Special Issue Advances in Technology-Enhanced Teaching and Learning)
Versions Notes

Abstract

:
Saturation is a key construct used by qualitative research theorists as a way to verify the rigor and validity of a study. This study intends to examine how saturation is used within qualitative educational technology studies that addressed saturation. We selected journals which (a) follow a peer review process; (b) are considered influential in their fields and of high quality, as reflected in being in Q1 (first quartile) or Q2 (second quartile) in Scopus; and (c) publish qualitative research. The number of journals selected for this study was 20, and we selected 3 articles from each. The methodology in the chosen articles was interview-based, and the articles were published in the journal in the 5 recent years 2018–2022 (i.e., a 5-year review period). The research results indicated that 20 out of 60 of the participating studies paid attention to the definition of saturation, 25 out of 60 participating studies paid attention to ‘why saturation was sought’, and 26 out of 60 participating studies paid attention to ‘when saturation was sought’. All the participating studies paid attention to ‘how was saturation assessed’. The most-mentioned expression to indicate the achievement of saturation was the emergence of new information, themes, categories, and codes, where most of the participating studies used expressions related to this aspect.

1. Introduction

Saturation has been considered by qualitative research theorists as an important construct that serves to verify the rigor and validity of the research [1], and thus the appropriateness of the emerging categories and themes to the studied phenomenon. Recently, researchers in social and educational research have been paying more attention to saturation, considering it an essential construct that needs to be present transparently in the research methodology of the qualitative research [2,3,4].
Glaser and Strauss developed the concept of saturation in the framework of a grounded theory approach to qualitative research. In grounded theory, the aim is to construct a theory from the inductive analysis of textual data to describe social or educational phenomena [5]. To ensure the validity of the inductive process of data collecting and analysis, saturation is suggested as a main means.
Researchers related saturation to two processes in the data collection and analysis processes: saturation of the set of categories related to a studied phenomenon, and saturation of the properties of a category found to be related to a studied phenomenon [6]. This is in line with the definition put forward by Glaser and Strauss [5] of saturation referring to the point in data collection and analysis when no additional themes and categories emerge from data, which happens when all relevant categories and sub-categories have been identified and described. We propose that the first part of the previous description is related to the saturation of the set of categories, while the second part of the previous description is related to the saturation of the properties of each category. In addition, Sharma et al. ([7], p. 7) describes the saturation of categories as: “saturate categories include the phase in which the researcher reaches a situation where he/she realizes that no more information is coming up for the category under study. Each category has its saturation limits and the same depends on the judgment of research”. Performing the two saturation processes sets the ground for the building of a comprehensive and credible theory that describes the studied educational phenomenon.
Defining saturation in one of two ways or in a definition that includes the two ways is present in various definitions of saturation by qualitative researchers. Bryant and Charmaz ([8], p. 611) defined theoretical saturation in terms that deliver both types of saturation as “the point at which gathering more data about a theoretical construct reveals no new properties nor yields any further theoretical insights about the emerging grounded theory”. In addition, Urquhart ([9], p. 194) defined saturation in terms of new codes: ‘the point in coding when you find that no new codes occur in the data. There are mounting instances of the same codes, but no new ones’, whilst Given ([10], p. 135) described saturation as the point at which ‘additional data do not lead to any new emergent themes’. A similar position regarding the (non)emergence of new codes or themes has been taken by others (e.g., [11,12]).
In addition, Saunders et al. [4] found that researchers follow four models of saturation. The first model is the theoretical saturation, which relates to the development of theoretical categories; related to grounded theory methodology. The second model is the inductive thematic saturation, which relates to the emergence of new codes or themes. The third model is the a priori thematic saturation; related to the degree to which identified codes or themes are exemplified in the data. The fourth model is the data saturation, related to the degree to which new data repeat what was expressed in previous data. Sebele-Mpofu [1] uses different studies to elaborate the previous saturation models of Saunders et al. [4].
Saturation could result from theoretical sampling. Breckenridge and Jones ([13], p. 2) describe theoretical sampling as constant comparison of codes: “Constant comparison of codes yields a provisional set of conceptual categories, from which point new categories emerge and new incidents are fitted and re-fitted into existing categories”. The researcher samples both for theoretical similarity and difference to expound the properties of each category, attempting to saturate all categories until the emergence of a core category [5]. Theoretical sampling is thereafter focused on data that is sufficiently and significantly relevant to the core category and its related properties. Moreover, Strauss and Corbin [14] described theoretical sampling as a means to “maximise opportunities to discover variations among concepts and to densify categories in terms of their properties and dimensions” (p.201). The previous argument relates theoretical sampling to saturation, where theoretical sampling ensures saturation. In addition, Saunders et al. [4] argue that the relatedness of sampling to saturation leads to the relatedness of saturation to the notion of theoretical sampling.
In addition to the above, saturation could be related to the sample size issue. Glaser and Strauss ([5], p. 61) describe this relatedness, when they address when to stop sampling; i.e., when to be satisfied with the sample size:
The criterion for judging when to stop sampling the different groups pertinent to a category is the category’s theoretical saturation. Saturation means that no additional data are being found whereby the sociologist can develop properties of the category. As he sees similar instances over and over again, the researcher becomes empirically confident that a category is saturated. He goes out of his way to look for groups that stretch diversity of data as far as possible, just to make certain that saturation is based on the widest possible range of data on the category.
Urquhart [9] and Birks and Mills [11] relate saturation primarily to the termination of analysis, rather than to the collection of new data. The termination of analysis indicates also the termination of data collection, i.e., being satisfied with the size of the sample already used to collect the data. Morse ([15], p. 587) argues that the most common leading principle for evaluating the sufficiency of a purposive sample is saturation: “Saturation is the most frequently touted guarantee of qualitative rigor offered by authors to reviewers and readers”. In addition, the saturation construct leads the researcher to combine sampling, data collection and data analysis, and thus not treating these components of qualitative research as separate stages in a linear process ([16], p. 18).
Researchers were interested in the types of saturation used in qualitative research papers. Hennink et al. [6] talked about two types of saturation: code saturation and meaning saturation. They described code saturation as indicating when researchers have “heard it all” and the range of thematic issues was identified, while meaning saturation indicates what is needed to “understand it all.” (p. 591). Examining 25 in-depth interviews, Hennink et al. [6] found that code saturation was reached at nine interviews, where, at the same time, 16 to 24 interviews were needed to reach meaning saturation.
Saunders et al. [4], drawing on a number of examples in the literature, explored three issues related to saturation: (1) ‘What?’—in what way(s) is saturation defined? (2) ‘Where and why?’—in what types of qualitative research, and for what purpose, should saturation be sought? and (3) ‘When and how?’—at what stage in the research is saturation sought, and how can we assess if it has been achieved? The results of the first issue revealed four ways of defining saturation (see above). The results of the second issue showed that where and why of the saturation depended on the way saturation was defined. Thus, in a deductive approach, saturation “may refer to the extent to which predetermined codes or themes are adequately represented in the data—rather like the idea of the categories being sufficiently replete with instances, or ‘examples’, of data, as suggested in the a priori thematic saturation model” (p. 1898). Saunders et al. [4] emphasized that ‘Where and why’ also depended on the type of research, where in narrative research, saturation fitted less. Here, narrative research is related to inquiry about the participants’ stories and telling the stories in a narrative way. The results of the third issue showed that ‘When and how?’ depended on the way saturation was defined. By using the concept of informational redundancy, the data saturation definition assumes that saturation should be assessed early in the process. Determination of the data saturation process precedes formal analysis. Some studies reported the use of saturation during the interview process. Considering the inductive thematic saturation, saturation is reached at the level of analysis, in relation to the non-emergence of new codes or themes. In the theoretical saturation, the question of considering saturation is at a more advanced stage and at a higher level of theoretical generality.

2. Research Rationale, Goals and Questions

The present research intends to verify how saturation was used as a component of the qualitative methodology in educational technology research. Doing that, it describes this use in papers that utilized saturation to describe the validity and reliability of their research. Vasileiou et al. [17] invited researchers to consider how saturation parameters, such as sample size, are found in prior studies. This invitation assumes that this consideration has a vital role in supporting transparent reporting. The present research will help researchers in the social sciences in general and researchers in educational technology in particular, in paying attention to an important methodological issue in qualitative research, i.e., saturation.
We will consider saturation in the present research by addressing the three saturation issues targeted by Saunders et al. [4] regarding saturation in qualitative research. These issues were: What, ‘where and why’, and ‘when and how’. We divided the third issue into two issues: the ‘when’ issue and the ‘how’ issue. In addition, we will consider an additional issue not emphasized in Saunders et al. [4], which is ‘what expressions were used’ to indicate that saturation was achieved? We put special emphasis on the additional topic as it has been little studied. The five research questions that the present research will try to answer are:
  • ‘What?’—in what way(s) is saturation defined?
  • ‘Where and why’ was saturation sought?
  • ‘When (at what stage in the research)’ is saturation sought, and how was it assessed?
  • ‘How’ is saturation achievement assessed?
  • ‘What expressions were used’ to indicate that saturation was achieved?
As this research intended to examine both theoretical as well as practical aspects of saturation, we referred to Saunders et al. [4]. As for the first issue, which is the way to define saturation, that pertains more to the theoretical side of saturation, while the other three have to do more with the practical side. Adding the saturation expression issue in the present study was a logical addition as it is expected to be in all the studies that address saturation, while the other issues could be found to some extent in the previous studies.

3. Methods

3.1. Search Strategy

Identifying studies was accomplished by utilizing the search function of each journal. We used the terms ‘technology’, ‘qualitative’, ‘interview*’, AND ‘saturation’. We limited the results to articles published between 1 January 2018 and 31 October 2022.

3.2. Eligibility: Inclusion/Exclusion Criteria

A structured search for articles related to educational technology and reporting interview-based qualitative studies was carried out and qualitative analysis was applied to the studies that met the criteria. We selected journals which:
6.
Follow a peer review process.
7.
Are considered influential in their fields and of high quality, as reflected in being in the first or second quartile in Scopus.
8.
Publish qualitative research. Twenty education journals were chosen.
9.
We chose three articles from each journal. These chosen articles were:
10.
Published in the journal in the five recent years 2018–2022 (i.e., a 5-year review period).
11.
Empirical studies.
12.
Are concerned with the integration of technology in education.
13.
Use interviews as a data collecting tool.

3.3. Data Saturation

When we arrived at 45 studies, we reached saturation in two meanings. The first meaning is the saturation of categories, where no new categories emerged. The second meaning is the properties of each category, where the properties of the categories did not change after analyzing 45 studies. In spite of this saturation, we continued to analyze another 15 studies from additional educational technology journals, where this analysis proved the saturation at which we arrived earlier.
In addition to the above, this number (60) constitutes a suitable number (above the average of 14) of studies for qualitative synthesis, where too many studies can overwhelm and exhaust the saturation point, leaving no additional insights [18]. For example, Hennink and Kaiser [3] used 23 studies to perform a systemic review of studies regarding the sample size they used for saturation. Lin et al. [19] focused on 22 empirical studies to study their interactive designs of oral tasks by evaluating the teaching methods, the types of oral tasks, the role played by the robots and the facilitators, and their effectiveness as a tool for improving oral competence. Here, we considered 60 studies as these studies were spread over five years.

3.4. Data Extraction and Analysis

To examine each of the issues in which the present research is interested, an inductive content analysis was initially conducted. Inductive content analysis processes consist of three phases, according to Elo et al. [20]: preparation, organization, and reporting of results. During the preparation stage, data is collected and sorted, and the unit of analysis is selected, while the organization phase includes open coding, creating categories, and abstraction. In the reporting phase, the phenomenon is described based on the content of the categories. Specifically for the present study, on the basis of this analysis, the categories that expressed qualitatively different values related to an issue were developed.
Table 1 shows the themes and codes related to each research question.

4. Results

The present research intended to verify five issues related to saturation in educational technology studies. Below, we address each issue, describing the categories related to it, and which emerged as a result of the analysis. Then we give an example of a study that satisfied the category.

4.1. ‘What?’—In What Way(s) Is Saturation Defined?

The reviews of the studies related to the use of technology in education showed that two values emerged in the answer to the first saturation issue. The first value is not defining saturation but describing how it was reached, while the second value is defining saturation and describing how it was reached. Specifically, the definition of saturation happened in four ways: addressing the set of categories, addressing the properties of categories, addressing the redundancy of information, and addressing objectivity.
Charmaz, ([21], p. 213) defines saturation as the situation when “you have defined, checked, and explained relationships between categories and the range of variation within and between your categories”.
Tomczyk and Walker ([22], p. 8) defined saturation by referring to the properties of a category: “Repeated statements were not included in a given category due to the saturation of a given area”. Graham ([23], p. 149) defined saturation by referring to the redundancy of information: “Saturation is the point in the data collection process where the information becomes redundant”. Reilly et al. ([24], p. 2241) defined saturation by referring to objectivity: “Therefore, the paper considers saturation to be in balance with objectivity”. Table 2 shows the frequency of the values of ‘in what way(s) saturation is defined’ among the participating papers.

4.2. Why Was Saturation Sought?

The reviews of the studies related to the use of technology in education showed that four values emerged in the answer to the second saturation issue, which is ‘why saturation was sought’. These values concerned using saturation to decide whether to continue the data collecting, whether to continue the combined process of data collecting and data analysis, whether to continue the data analysis, and whether to consider codes and categories as reliably identifiable.
Bilgiç and Tuzun [25] used saturation to decide whether to continue the data collecting, and specifically to decide on the transcript to continue the coding with. Lewis et al. ([26], p. 959) sought saturation to decide whether to continue the combined process of data collecting and data analysis: “We conducted data analysis concurrently with data collection until we reached saturation”. In addition, Lovrić et al. ([27], p. 3) used saturation to decide whether to continue the data analysis process: “The sample size was further determined based on informational needs and using a model of inductive thematic saturation, as a criterion for discontinuing data analysis”. Reilly et al. ([24], p. 2241) used saturation in the context of deciding whether to consider codes and categories as reliably identifiable: “The paper considers saturation …. to be achieved when the themes that can be mined from the paper are those that are reliably identifiable”. In the previous citation, Reilly et al. ([24], consider saturation to be sought in order to ensure that the resulting codes are an adequate summary of coded phenomenon.
Table 3 shows the frequency of the values of why’ saturation was sought’ among the participating papers.

4.3. When (at What Stage in the Research) Is Saturation Sought?

The reviews of the studies related to the use of technology in education showed that three values emerged regarding when saturation is sought: addressing saturation at the collecting data phase, addressing saturation at the data analysis phase, and addressing saturation at both the data collecting and analysis phases.
Part of the researchers were engaged with addressing saturation at the collecting data phase. Graham ([23], p. 149) described saturation as: “Saturation is the point in the data collection process where the information becomes redundant”. Part of the researchers started addressing saturation at the data analysis phase. Krasny et al. [28] kept looking for saturation in the analysis phase: “The second author used an open-coding strategy in which he wrote memos regarding themes as they emerged from the texts and organized them into categories that represented overlapping themes, using an iterative process until saturation was reached among themes”. Here, saturation began at the analysis phase as the authors talk about generation of codes and categories.
Part of the researchers combined the collecting data and analysis data phases. Lewis et al. [26] combined the data collecting and analysis processes to examine the achievement of saturation. They began examining saturation as they started collecting data and to do so, they kept analysing the data they collected (see above). Table 4 shows the frequency of the values of ‘when saturation was sought’ among the participating papers.

4.4. How Was Saturation Assessed?

The reviews of the studies related to the use of technology in education showed that three values emerged regarding how saturation is sought: assessing saturation by considering the emergence of new information, themes, categories, codes, etc; assessing saturation by considering the range of themes and categories; and assessing saturation by considering the interviewing process. Below, we describe these values, without giving examples on them, as the reader can find relevant examples when describing the next issue; that is, expressions that were used to indicate that saturation was achieved.
A selection of the participating studies reported the assessment of saturation through considering the emergence of new information or insight. In doing that, they used different expressions, where these expressions were general as information and insight, or more specific as themes and categories.
A selection of the participating studies reported the assessment of saturation through considering the range of themes and categories. In doing so, they used the expressions ‘range of themes’, ‘range of terms’, and ‘the sufficiency of data’.
A selection of the participating studies reported the assessment of saturation through considering the interviewing process. In doing so, they used the expressions ‘interviewing process’, ‘reliability of the process of arriving at categories’, and ‘participants’ knowledge and experience’.
Table 5 shows the frequency of the values of ‘how was saturation assessed’ among the participating papers.

4.5. Expressions That Were Used to Indicate That Saturation Was Achieved

The description, of ‘what indicated that saturation was reached’, was expressed using 14 terms: the sample size, new information, new insights/recurring insight, new concepts, new codes, new themes, the number of participants, the number of interviews, the number of codes and categories arrived at, the range of themes, the sufficiency of data, the process of interviewing, the reliability of the themes, and participants’ knowledge and experience. Below, we address each case.

4.5.1. Expressions Used When the Assessing of Saturation Was Done by Considering the Emergence of New Information, Themes, Categories, Codes, Etc

Referring to the sample size:
Al-Kumaim et al. ([29], p. 4) described the relatedness of reaching saturation to the research sample size: “As in qualitative research, sample sizes should not be large as this will prevent the extraction of enough rich data. At the same time, if a sample is too small, it will be hard to reach data saturation”. Thus, Al-Kumaim et al. [29] did not specify a sample size as indicating saturation, but talked about a size that is neither big nor small.
Referring to new information:
Mirmoghtadaie et al. ([30], p. 63) described how saturation was reached in terms of new information: “There appeared to be saturation of data as no new information was gained from the last interviews.” Zhu and Bonk [31] used the term ‘limited new information’ to describe how saturation was reached: “The data reached a saturation point after 22 interviews, as there was limited new information identified at that point of the interview process” (p. 111).
Referring to new codes:
Kara Aydemir and Can ([32], p. 1092) described achieving saturation by referring to new codes: “it was observed that the final three interview transcripts provided few or no new codes. Therefore, it was determined that saturation had been achieved.”
Referring to new themes:
Cooper et al. ([33], p. 5) described achieving saturation by referring to new themes: “Saturation point was reached when no new themes emerged from the data”. Authors also talked about recurring themes instead of the absence of new themes.
Referring to the number of participants:
Reedy ([34], p. 135) described achieving saturation by referring to the number of participants: “Given that most qualitative researchers find that data saturation is often reached with about 20 participants when conducting in-depth interview-based studies …, the sample size of 19 is considered the optimum number for this study”.
Referring to the number of interviews:
Zhu and Bonk ([31], p. 109) described achieving it in terms of the number of interviews: “In this study, the researcher found that the data had reached saturation after finishing 22 instructor interviews”. Some studies described achieving saturation in terms of reports, where the report was the unit of data collection. Lovrić et al. ([27], p. 1) described saturation when they used participants’ reports as data collection: “data saturation was achieved after analyzing the reports of 33 undergraduate nursing students”.
Referring to the number of codes and categories:
Bilgiç and Tuzun ([25], p. 148) described achieving saturation by mentioning the number of codes and categories that they achieved:
The category saturation was reached when a total of 727 codes were generated under 13 categories by the total coding process. Analysis continued with the selective coding in which researchers checked for the repeating codes and categories and organized the relationships between the codes and categories. In the end, 608 codes were obtained in 9 categories which represented the core issues.

4.5.2. Expressions Used When the Assessing of Saturation Was Done by Considering the Range of Themes and Categories

Referring to the range of themes:
Crook and Nixon ([35], p. 8) described achieving saturation by referring to the range of themes: “Finally, the findings reported here are based on a sample of only modest scale. Yet we believe that it was large enough to reach a saturation point in relation to the range of themes active in relation to characterising the student experience”
Referring to the sufficiency of data:
Veletsianos et al. ([36], p. 1716) described achieving saturation by referring to the sufficiency of data: “… and then continued scheduling interviews until we felt confident that the data collected would be sufficient to answer the posed research question”.
Expressions used when the assessing of saturation was done by considering the interviewing process:
Referring to the process of interviewing:
Bruggeman et al. ([37], p. 4) described achieving saturation as a result of conducting the last interview several months after the first interview: “In addition, the twelfth and last interview was conducted several months after the first interview in order to improve stability of observations and reach saturation”.
Referring to the reliability of the themes:
Reilly et al. ([24], p. 2241)) described the achievement of saturation in terms of reliable identification of themes: “to be achieved when the themes that can be mined from the paper are those that are reliably identifiable”.
Referring to the participants’ knowledge and experience:
Interestingly, some researchers talked about not reaching saturation, and why they did not reach it: “Although the number of statements analysed is significant, in the applied research model it is not possible to speak of a full saturation of the categories delineated here, since many teachers likely do not know about these online discussion groups, or if they do, refrain from participating for whatever reason” [22]. Thus, for Tomczyk and Walker [22], the reason for not reaching saturation is the insufficient knowledge or experience of the participants. Tomczyk and Walker ([22], p. 27) defined saturation as they came to define how it was not reached: “it is not possible to speak of a full saturation of the categories delineated here”.
Table 6 shows the frequency of the values of ‘What expressions were used to indicate that saturation was achieved’ among the participating papers.

5. Discussion

The present research was interested in three issues related to saturation as described in papers related to technology use in education, published in the years 2019–2022, and published in journals that are ranked Q1 and Q2 in Scopus.

5.1. What Way(S) Was Saturation Defined?

The first issue in which the present research is interested was the ways saturation is defined. One-third of the studies paid attention to the definition of saturation (20 out of 60). Almost half of those (11 out of 20) defined it in terms of the set of categories, i.e., the dimensions issue of the categories [38]. This would indicate the concern of the researchers that the set of categories emerging from the data cover all the elements of the studied educational technology phenomenon. This coverage aspect lies behind another definition found in four of the participating papers, that of redundancy/sufficiency of information. This interest in the coverage points at qualitative researchers’ awareness that qualitative research seeks to understand how subjects interact with the social world through multiple and varied meanings [39], where this multiplicity needs varied categories to cover the studied phenomenon in which the research subjects are engaged.
Moreover, 4 of the papers defined saturation in terms of the properties of categories. Usually, researchers are concerned with the development of categories, through looking for their properties, for finding the relations between the categories ([40], p. 350), which could help with the development of the theory [41]. The reason for not mentioning the development of the categories in the definition of the saturation could be a result of being concerned with the extent of categories as a collective and not the extent of the individual category.
In addition to the above, very few participating papers mentioned objectivity in their definition of saturation. The reason could be the complexity of the definition of objectivity. For example, Eisner [42] argues that it is difficult to evaluate the correspondence between our views of reality and reality itself, which makes evaluating this correspondence a difficult matter.

5.2. Why Was Saturation Sought?

The second issue in which the present research was interested is ‘why saturation was sought’. Less than half of the participating studies referred to this issue (25 out of 60). A little less than half of the referring studies (12 out of 25) mentioned they used saturation to decide whether to continue the data collecting, while almost one-third of the studies (8 out of 25) mentioned that they used saturation to decide whether to continue the data analysis. Some of those that referred to the data collection phase could also have been engaged with the saturation issue in the data analysis phase, but they did not mention it.
The above research results indicate two points. First, part of the participating studies (here, more than half of them) are not concerned with saturation as a tool for deciding whether to continue/quit the collection or analyzing of the data. For them, it is sufficient to declare how they arrived at saturation, an issue that we will address in the context of the fourth research question. The first issue could indicate that some researchers are concerned with the practical issue of saturation, where it is enough to declare the method of arriving at saturation without declaring why this issue was addressed in the first place.
Second, most of the participating studies that referred to the issue of ‘why they sought saturation’ (22 out of 25) did that to decide upon quitting/continuing the data collection and analysis processes. This use of saturation could indicate considering saturation as a technique for deciding upon continuing or quitting data collection and analysis [43]. Thus, saturation here served as a technique in the process of performing the qualitative research, a technique that helped make decisions regarding this process.

5.3. When Was Saturation Sought?

The third issue in which the present research was interested is ‘when saturation was sought’. A little less than half of the participating studies referred to this issue (26 out of 60). More than half of the referring studies (14 out of 26) mentioned that they addressed saturation in the data analysis phase. This is in line with researchers who suggest that saturation can happen during or at the end of the analysis phase. Guest et al. ([2], p. 2) suggested a way to report saturation in qualitative research: “We subsequently propose an alternative way of evaluating saturation and offer a relatively easy-to-use method of assessing and reporting on it during or after an inductive thematic analysis”. For more description of the method please refer to the paper of Guest et al. [2]. It should be noted that data analysis is the final process before the writing-up of results, so arriving at saturation is a crucial step to decide the commencement of the results wite-up. Thus, saturation is an integrative block, not only of the methodology but also for the study writing itself; specifically the beginning of this writing.
The other participating studies mentioned that they addressed saturation in the data collection and analysis phases, or in the data collection phase. Considering research on saturation, part of the researchers talked about this process as part of the collection and analysis phases, though they did that only implicitly. Morse ([43], p. 1123) considered saturation “the phase of qualitative data analysis in which the researcher has continued sampling and analyzing data until no new data appear and all concepts of the theory are well-developed….and their linkages to other concepts are clearly described”. Morse began the description of saturation by considering it as part of the analysis phase but continued to consider it part of the sampling phase, i.e., the data collection phase. Here, the role of saturation is a thorough one where it is present all over the research process. It is in fact part of the cyclic process: collect, analyze, look for saturation. If saturation is achieved, stop the cyclic process, or continue it otherwise.

5.4. How Was Saturation Assessed?

The fourth issue in which the present research was interested is ‘how was saturation assessed’. All the participating studies referred to this issue (60 out of 60), where most of the referring studies (46 out of 60) mentioned that they assessed saturation by considering the emergence of new information, themes, categories, codes, etc. This result indicates that the studies were concerned with saturation as arriving at the whole set of categories that characterize the specific educational technology phenomenon. This aspect is a principal characteristic of saturation [7,8]. It is worth noting that the studies did not assess saturation by looking at the properties of categories, though some of them did define saturation as thus. The previous argument indicate that the participating studies looked at the educational phenomenon as a whole, and less at the local aspects of each category, where we look at properties as the conditions of the category occurrence, and the results of this occurrence.

5.5. What Expressions Were Used to Indicate That Saturation Was Achieved?

The fifth issue in which the present research was interested is ‘what expressions were used to indicate that saturation was achieved?’. All the participating studies referred to this issue (60 out of 60), which is why they were chosen to participate in the present study. This referring was done by referring to three main expressions. The first expression is the emergence of new information, themes, categories, and codes, where most of the participating studies used expressions related to this aspect. This use is in line with different studies that described saturation using this expression (ex., [9,10,11,12]), starting from Glaser and Strauss [5]. The terms within the first expression could be related to defining saturation in terms of categories or the properties of a category. It could be that the mentioning of these terms substituted giving definition to saturation.
Seven of the participating studies used the expression ‘the range of themes’ to describe saturation. Here, the participating studies were concerned with the properties of the individual categories more than in other means of saturation. The expression ‘range’ was also used by Glaser and Strauss [5], but to indicate the range of data in a category, where this range is sought by the researcher to ensure diversity of data “just to make certain that saturation is based on the widest possible range of data on the category” (p. 61).
A further 7 of the participating studies used the expression of the ‘interviewing process’ to indicate arriving at saturation. These studies were interested mainly in the reliability of the categorization of the data, where they assumed that what added to this reliability is the process of this categorization and those who were engaged in this process. Here, the participating studies were concerned mainly with the trustworthiness of the research. This trustworthiness would indicate confidence of the validity of the whole research process.

6. Conclusions, Limitations and Recommendations

The present research intended to examine issues related to saturation in qualitative educational technology studies. The contribution of the present research should be considered in light of the spread of educational technology in the classroom and the publication of a large quantity of papers on this issue. Research related to the integration of educational research now address the different educational aspects as the cognitive aspect [44], the affective aspect [45], the social aspect [46], the behavioural aspect [47] and the metacognitive aspect [48]. The results of the present research would show authors of educational technology papers, that address the different aspects of learning, how to address the issue of saturation in their papers and to include sufficient information about it.
The results of the present research indicate that not all the issues were transparent in the participating studies. Almost half the participating studies paid attention to the definition of saturation, while half of them paid attention to ‘why saturation was sought’. A little more than half the participating studies paid attention to ‘when saturation was sought’. All the previous results indicate the need for more transparency in reporting issues related to saturation in educational technology studies that use interviews in qualitative research [17].
It was suggested by Golafshani [49] that consistency of data could be achieved when raw data and data reduction were examined and verified during the research process. Data reduction could be examined through paying attention to the different issues of saturation that were disrobed and discussed in the present research. Studies that took care of the issues examined in the present research could be pointed at as achieving consistency of data, which has not been observed in more than half of the participating studies. Thus, as was recommended above, educational technology studies are called to encourage the consistency of their data and findings by taking care of the different issues of saturation with which the present research was concerned.
Some of the themes related to saturation that emerged in the present research could be the subject for discussion or dispute, such as the number of participants. In the present research, our main goal was to present a picture of what is happening in technology-based educational research papers. Theoretical research should address the issue of the fitness of expressions or ideas to the methodological notions that are at the heart of qualitative methodology. One of these potential expressions is the number of participants. It is worth noting that the number of participants or interviews was mentioned in the studied papers to express the arriving at saturation, but was not the reason for it.
The present research took care of educational technology studies, from Q1 or Q2, that were published in the five recent years 2018–2022. It is interesting to verify the five issues of saturation in qualitative studies from other rank quarters, as well as those that will be published in the future in the same quarters.
The present study took care of saturation issues presented in Saunders et al. [4], in addition to a fourth related issue, namely, ‘what expressions were used’ to indicate that saturation was achieved. We used in the present research the saturation issues in Saunders et al. [4], as these issues relate to theoretical as well as practical aspects of saturation. Future research could study additional issues related to saturation, such as theoretical sampling – for example, when and why researchers use saturation sampling and whether this use fits the features of theoretical sampling as described by researchers in qualitative methods. Another issue related to saturation is taking into consideration the variation of school types or the variations of age groups that the studies address.
In addition to the above, the present study sample included studies that included the term ‘saturation’ as a qualitative tool to ensure the validity of their data collection and analysis. We did not condition the inclusion of the study by its satisfaction of all the saturation issues addressed in the present research, as this was one of the aspects in which the present research was interested. Future research could use interviews with authors to investigate when and why they decide to take care of each saturation issue addressed in the present research. Moreover, we wanted to make a balance between three issues: the number of journals that we consider, the number of studies in a journal, and arriving at saturation. Although this paper did not want to evaluate the journal, but rather the saturation issues in educational technology papers, it was nonetheless important for us that this evaluation is not restricted to a select few journals. At the same time, we thought that one or two studies in a journal would not shed light on the saturation issue in that journal. Saturation of categories and the properties of categories was the leading principle about the overall number of studies that were chosen for analysis.

Funding

This research received no external funding.

Conflicts of Interest

The author declares no conflict of interest.

References

  1. Sebele-Mpofu, F.Y. Saturation controversy in qualitative research: Complexities and underlying assumptions. A Lit. Rev. Cogent Soc. Sci. 2020, 6, 1838706. [Google Scholar]
  2. Guest, G.; Namey, E.; Chen, M. A simple method to assess and report thematic saturation in qualitative research. PLoS ONE 2020, 15, e0232076. [Google Scholar] [CrossRef] [PubMed]
  3. Hennink, M.; Kaiser, B.N. Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Soc. Sci. Med. 2021, 292, 114523. [Google Scholar] [CrossRef]
  4. Saunders, B.; Sim, J.; Kingstone, T.; Baker, S.; Waterfield, J.; Bartlam, B.; Burroughs, H.; Jinks, C. Saturation in qualitative research: Exploring its conceptualization and operationalization. Qual. Quant. 2018, 52, 1893–1907. [Google Scholar] [CrossRef] [PubMed]
  5. Glaser, B.G.; Strauss, A.L. Discovery of Grounded Theory: Strategies for Qualitative Research; Routledge: Abingdon, UK, 2017. [Google Scholar]
  6. Hennink, M.; Kaiser, B.N.; Marconi, V.C. Code saturation versus meaning saturation: How many interviews are enough? Qual. Health Res. 2017, 27, 591–608. [Google Scholar] [CrossRef] [PubMed]
  7. Sharma, G.; Kulshreshtha, K.; Bajpai, N. Getting over the issue of theoretical stagnation: An exploration and metamorphosis of grounded theory approach. Qual. Quant. 2021, 56, 857–884. [Google Scholar] [CrossRef]
  8. Bryant, A.; Charmaz, K. (Eds.) The SAGE Handbook of Grounded Theory; Sage: London, UK, 2007. [Google Scholar]
  9. Urquhart, C. Grounded Theory for Qualitative Research: A Practical Guide; Sage: London, UK, 2013. [Google Scholar]
  10. Given, L.M. 100 Questions (and Answers) about Qualitative Research; SAGE Publications: London, UK, 2015. [Google Scholar]
  11. Birks, M.; Mills, J. Grounded Theory: A Practical Guide; Sage Publishing: London, UK, 2015. [Google Scholar]
  12. Olshansky, E.F. Generating theory using grounded theory methodology. In Nursing Research Using Grounded Theory: Qualitative Designs and Methods in Nursing; De Chesnay, M., Ed.; Springer: Berlin/Heidelberg, Germany, 2015; pp. 19–28. [Google Scholar]
  13. Breckenridge, J.; Jones, D. Demystifying theoretical sampling in grounded theory research. Grounded Theory Rev. 2009, 8, 112–126. [Google Scholar]
  14. Strauss, A.; Corbin, J. Basics of Qualitative Research Techniques; Sage Publications: London, UK, 1998. [Google Scholar]
  15. Morse, J. Data were saturated. Qual. Health Res. 2015, 25, 587–588. [Google Scholar] [CrossRef] [Green Version]
  16. Bryman, A. Many Qualitative Interviews is Enough? Expert Voices and Early Career Reflections on Sampling and Cases in Qualitative Research; Baker, S.E., Edwards, R., Eds.; National Centre for Research Methods: Southampton, UK, 2012; pp. 18–20. [Google Scholar]
  17. Vasileiou, K.; Barnett, J.; Thorpe, S.; Young, T. Characterising and justifying sample size sufficiency in interview-based studies: Systematic analysis of qualitative health research over a 15-year period. BMC Med. Res. Methodol. 2018, 18, 1–18. [Google Scholar] [CrossRef] [Green Version]
  18. Finfgeld-Connett, D. Use of content analysis to conduct knowledge-building and theory-generating qualitative systematic reviews. Qual. Res. 2014, 14, 341–352. [Google Scholar] [CrossRef]
  19. Lin, V.; Yeh, H.C.; Chen, N.S. A Systematic Review on Oral Interactions in Robot-Assisted Language Learning. Electronics 2022, 11, 290. [Google Scholar] [CrossRef]
  20. Elo, S.; Kääriäinen, M.; Kanste, O.; Pölkki, T.; Utriainen, K.; Kyngäs, H. Qualitative content analysis: A focus on trustworthiness. SAGE Open 2014, 4, 2158244014522633. [Google Scholar] [CrossRef]
  21. Charmaz, K. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis, 2nd ed.; SAGE: London, UK, 2014. [Google Scholar]
  22. Tomczyk, Ł.; Walker, C. The emergency (crisis) e-learning as a challenge for teachers in Poland. Educ. Inf. Technol. 2021, 26, 6847–6877. [Google Scholar] [CrossRef]
  23. Graham, A. Benefits of online teaching for face-to-face teaching at historically black colleges and universities. Online Learn. 2019, 23, 144–163. [Google Scholar]
  24. Reilly, C.M.; Kang, S.Y.; Grotzer, T.A.; Joyal, J.A.; Oriol, N.E. Pedagogical moves and student thinking in technology-mediated medical problem-based learning: Supporting novice-expert shift. Br. J. Educ. Technol. 2019, 50, 2234–2250. [Google Scholar] [CrossRef]
  25. Bilgiç, H.G.; Tuzun, H. Issues and challenges with web-based distance education programs in Turkish higher education institutes. Turk. Online J. Distance Educ. 2020, 21, 143–164. [Google Scholar] [CrossRef] [Green Version]
  26. Lewis, D.G.R.; Gerber, E.M.; Carlson, S.E.; Easterday, M.W. Opportunities for educational innovations in authentic project-based learning: Understanding instructor perceived challenges to design for adoption. Educ. Technol. Res. Dev. 2019, 67, 953–982. [Google Scholar] [CrossRef]
  27. Lovrić, R.; Farčić, N.; Mikšić, Š.; Včev, A. Studying during the COVID-19 pandemic: A qualitative inductive content analysis of nursing students’ perceptions and experiences. Educ. Sci. 2020, 10, 188. [Google Scholar] [CrossRef]
  28. Krasny, M.E.; DuBois, B.; Adameit, M.; Atiogbe, R.; Alfakihuddin, M.L.; Bolderdene, T.; Golshani, Z.; González-González, R.; Kimirei, I.; Leung, Y.; et al. Small Groups in a Social Learning MOOC (sIMOOC): Strategies for Fostering Learning and Knowledge Creation. Online Learn. 2018, 22, 119–139. [Google Scholar] [CrossRef]
  29. Al-Kumaim, N.H.; Alhazmi, A.K.; Mohammed, F.; Gazem, N.A.; Shabbir, M.S.; Fazea, Y. Exploring the impact of the COVID-19 pandemic on university students’ learning life: An integrated conceptual motivational model for sustainable and healthy online learning. Sustainability 2021, 13, 2546. [Google Scholar] [CrossRef]
  30. Mirmoghtadaie, Z.; Ahmady, S.; Kohan, N.; Rakhshani, T. Explaining the Concept and Dimensions of Professional Functions in Online Learning System of Medical Sciences: A Qualitative Content Analysis. Turk. Online J. Distance Educ. 2019, 20, 61–72. [Google Scholar] [CrossRef]
  31. Zhu, M.; Bonk, C.J. Designing MOOCs to Facilitate Participant Self-Monitoring for Self-Directed Learning. Online Learn. 2019, 23, 106–134. [Google Scholar] [CrossRef] [Green Version]
  32. Kara Aydemir, A.G.; Can, G. Educational technology research trends in Turkey from a critical perspective: An analysis of postgraduate theses. Br. J. Educ. Technol. 2019, 50, 1087–1103. [Google Scholar] [CrossRef]
  33. Cooper, V.A.; Forino, G.; Kanjanabootra, S.; Von Meding, J. Leveraging the community of inquiry framework to support web-based simulations in disaster studies. Internet High. Educ. 2020, 47, 100757. [Google Scholar] [CrossRef]
  34. Reedy, A.K. Rethinking online learning design to enhance the experiences of Indigenous higher education students. Australas. J. Educ. Technol. 2019, 35, 132–149. [Google Scholar] [CrossRef] [Green Version]
  35. Crook, C.; Nixon, E. How internet essay mill websites portray the student experience of higher education. Internet High. Educ. 2021, 48, 100775. [Google Scholar] [CrossRef]
  36. Veletsianos, G.; Johnson, N.; Belikov, O. Academics’ social media use over time is associated with individual, relational, cultural and political factors. Br. J. Educ. Technol. 2019, 50, 1713–1728. [Google Scholar] [CrossRef]
  37. Bruggeman, B.; Tondeur, J.; Struyven, K.; Pynoo, B.; Garone, A.; Vanslambrouck, S. Experts speaking: Crucial teacher attributes for implementing blended learning in higher education. Internet High. Educ. 2021, 48, 100772. [Google Scholar] [CrossRef]
  38. Vollstedt, M.; Rezat, S. An Introduction to grounded theory with a special focus on axial coding and the coding paradigm. In Compendium for Early Career Researchers in Mathematics Education; ICME-13 Monographs; Kaiser, G., Presmeg, N., Eds.; Springer: Cham, Switzerland, 2019. [Google Scholar] [CrossRef] [Green Version]
  39. Silva, D.C.D.; Martins Júnior, F.R.F.; Silva, T.; Ribeiro, M.; Nunes, J.B.C. The characteristics of qualitative research: A study with theses from a postgraduate program in education. Educ. Rev. 2022, 38. Available online: https://www.scielo.br/j/edur/a/vfYpxdKhR6BBSrf3YpSHjqz/?lang=en&format=pdf (accessed on 10 January 2023). [CrossRef]
  40. Suter, W.N. Introduction to Educational Research: A Critical Thinking Approach, 2nd ed.; SAGE Publications: London, UK, 2012. [Google Scholar] [CrossRef]
  41. Cohen, L.; Manion, L.; Morrison, K. Research Methods in Education, 6th ed.; Routledge: Abingdon, UK, 2007. [Google Scholar]
  42. Eisner, E. The Enlightened Eye: Qualitative Inquiry and the Enhancement of Educational Practice; Macmillan: New York, NY, USA, 1991. [Google Scholar]
  43. Morse, J.M. Theoretical saturation. In The Sage Encyclopedia of Social Science Research Methods; Lewis-Beck, M.S., Bryman, A., Liao, T.F., Eds.; Sage: Thousand Oaks, CA, USA, 2004; p. 1123. Available online: http://sk.sagepub.com/reference/download/socialscience/n1011.pdf (accessed on 10 January 2023).
  44. Daher, W. Preservice teachers’ perceptions of applets for solving mathematical problems: Need, difficulties and functions. J. Educ. Technol. Soc. 2009, 12, 383–395. [Google Scholar]
  45. Daher, W. Discursive positionings and emotions in modelling activities. Int. J. Math. Educ. Sci. Technol. 2015, 46, 1149–1164. [Google Scholar] [CrossRef]
  46. Daher, W. Mathematics learning community flourishes in the cellular phone environment. Int. J. Mob. Blended Learn. IJMBL 2010, 2, 1–7. [Google Scholar] [CrossRef]
  47. Abuzant, M.; Ghanem, M.; Abd-Rabo, A.; Daher, W. Quality of using google classroom to support the learning processes in the automation and programming course. Int. J. Emerg. Technol. Learn. iJET 2021, 16, 72–87. [Google Scholar] [CrossRef]
  48. Daher, W.; Baya’a, N.; Jaber, O.; Awawdeh Shahbari, J. A Trajectory for Advancing the Meta-Cognitive Solving of Mathematics-Based Programming Problems with Scratch. Symmetry 2020, 12, 1627. [Google Scholar] [CrossRef]
  49. Golafshani, N. Understanding Reliability and Validity in Qualitative Research. Qual. Rep. 2003, 8, 597–606. [Google Scholar] [CrossRef]
Table
Table 1. Themes and codes related to each research question.
Table 1. Themes and codes related to each research question.
Research Question CategoryCodesExamples
‘What?’—in what way(s) is saturation defined?‘Saturation is’, ‘Saturation addresses’, ‘Saturation is defined as’.Saturation addresses the coverage of the educational phenomenon by the set of categories emerging at the data analysis.
Why was saturation sought?‘We performed saturation to’, ‘Saturation was sought to’, ‘Saturation was considered in order’.Saturation was sought to decide whether to continue the data collecting.
When (at what stage in the research) is saturation sought?‘Saturation was sought when’, ‘Saturation was performed during’, ‘Saturation was considered at the stage of’. The researchers addressed saturation at the data analysis phase.
How was saturation assessed?‘Saturation was achieved when’, ‘What indicated saturation was’, ‘Saturation was assessed by’. Saturation was achieved when the range of categories arrived at indicated that the educational phenomenon was described sufficiently.
Expressions that were used to indicate that saturation was achieved‘No new information emerged’, ‘No new codes emerged’, ‘No new themes emerged’. ‘We knew that saturation was achieved when no new coded emerged from the analysis of the interviews’.
Table
Table 2. Values of ‘in what way(s) saturation is defined’.
Table 2. Values of ‘in what way(s) saturation is defined’.
Value of Way of SaturationFrequency
In terms of the set of categories11
In terms of the properties of categories 4
In terms of the redundancy/sufficiency of information4
In terms of objectivity1
Table
Table 3. Values of ‘why’ saturation was sought’.
Table 3. Values of ‘why’ saturation was sought’.
Value of Why Saturation Was SoughtFrequency
To decide whether to continue the data collecting 12
To decide whether to continue the combined process of data collecting and data analysis 2
To decide whether to continue the data analysis 8
To decide whether to consider codes and categories as reliably identifiable3
Table
Table 4. Values of ‘when saturation was sought’.
Table 4. Values of ‘when saturation was sought’.
Value of When Saturation Was SoughtFrequency
Addressing saturation at the collecting data phase 7
Addressing saturation at the data analysis phase 14
Addressing saturation at both the data collecting and analysis phases.5
Table
Table 5. Values of ‘how was saturation assessed’.
Table 5. Values of ‘how was saturation assessed’.
Value of ‘How Was Saturation Assessed’Frequency
Assessing saturation by considering the emergence of new information, themes, categories, codes, etc.46
Assessing saturation by considering the range of themes 7
Assessing saturation by considering the interviewing process7
Table
Table 6. Frequency of values of ‘What expressions were used to indicate that saturation was achieved’.
Table 6. Frequency of values of ‘What expressions were used to indicate that saturation was achieved’.
Expressions of ‘When Saturation Was Achieved’Frequency
Assessing saturation by considering the emergence of new information, themes, categories, codes, etc:
The sample size 16
No new information 6
No new insights 1
No new concepts3
No new codes 6
No new themes 4
The number of participants 3
The number of interviews 4
The number of codes and categories 3
Assessing saturation by considering the range of themes
The range of themes 6
The sufficiency of the data1
Assessing saturation by considering the interviewing process
The process of interviewing 1
The process of arriving at categories 4
Participants’ knowledge and experience 2
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Daher, W. Saturation in Qualitative Educational Technology Research. Educ. Sci. 2023, 13, 98. https://doi.org/10.3390/educsci13020098

AMA Style

Daher W. Saturation in Qualitative Educational Technology Research. Education Sciences. 2023; 13(2):98. https://doi.org/10.3390/educsci13020098

Chicago/Turabian Style

Daher, Wajeeh. 2023. "Saturation in Qualitative Educational Technology Research" Education Sciences 13, no. 2: 98. https://doi.org/10.3390/educsci13020098

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop