1. Making Sense of Digital Discourse: A Demonstration of Automated Textual Analysis
David A. Thomas
Mathematics Education Associates LLP
Great Falls, MT USA
Jeanine Mwambakana
University of Pretoria
Pretoria, South Africa
The Problem
Many scholars rely on information technologies to search for and retrieve records and publications pertinent to their research interests. But what is a scholar to do when a search identifies hundreds of documents, any of which might be vital or irrelevant to his/her work? Automated technologies are now available for characterizing the global features of massive document sets and for identifying relationships between concepts and/or themes and the individuals and organizations associated with them.
Content vs. Structure
To understand massive unstructured textual data sets, one must attend to both its content and structure. Content analysis focuses on the substance of textual data (e.g., What are the common themes in a set of papers or blog postings?). Social network analysis focuses on the structure of information and/or communication (e.g., Who are the emergent leaders in an online forum?). This poster focuses on the goals, methodology, and technology of automated textual analysis.
Research Questions
What are the emergent concepts/themes in the mathematics education literature associated with teaching and learning mathematical modelling?
How do papers addressing mathematical modelling at the secondary and tertiary levels differ? Which concepts/themes are found at both levels?
Which papers/authors at both levels are closely related in terms of concepts/themes? Which have little in common?
Data & Documents
Online databases were used to identify a representative sample of frequently cited journal articles focused on modelling at the secondary (15) and tertiary (21) levels.
Documents were then sorted into two folders, high school (HS) and tertiary (T).
Document formats read include most commonly used file extensions (e.g., *.docx, *.doc, *.rtf, *.pdf, *.txt, *.xlsx, and so on)
Methodology & Technology
Goals of automated content analysis
Facilitate analysis of document sets unfettered by a priori assumptions or theoretical frameworks used by the researcher, consciously or unconsciously, as a scaffold for the identification of concepts and themes in the data
Reduce time & costs associated with analysis of massive document sets
Facilitate more rapid and frequent analysis and reanalysis of text
Supporting technology
Leximancer,
Think of concepts as “collections of words that typically travel together throughout the text”. For example, in a document about climate change, the Leximancer concept carbon might include the keywords dioxide, carbonate, footprint, and sequester. Leximancer weights these terms according to how frequently they occur in sentences containing the concept, compared to how frequently they occur elsewhere. Leximancer induces the definition of each concept (i.e. the set of weighted terms) by noting the co-occurrence of words within a “sliding window” as it scans blocks of text a few sentences at a time.
These data are used to make two determinations: (i) the most frequently used concepts within a body of text; and more importantly, (ii) the relationships between these concepts (e.g., the co‐occurrence between concepts). During the learning process, words highly relevant to the seed are continuously updated, and eventually form a thesaurus of terms for each concept.
Emergent Themes
Knowledge Pathway Between Concepts
Knowledge Pathway Between High School & Tertiary Papers
Discussion
Broader Implications
Individual scholars may create digital folders of seminal documents in their research domains and analyse the entire corpus of work in terms of concepts, themes, authors, and their connections. In this approach, both the forest and its boundaries can be contemplated holistically.
Student research proposals may be analysed relative to a particular knowledge domain and to motivate questions such as, “How are your research questions grounded in the literature? Are your research design, sampling, and data analysis procedures consistent with accepted practice?”
Teams of scholars working within or across disciplines can more readily share their respective interests and experiences, juxtapose their potential contributions, and identify challenges on the boundaries and at the intersections of their fields.