Short description of application purpose (advantages, disadvantages)

Documentary Analysis using Qualitative Content Analysis (here abbreviated DocA-CA) is a method for the systematic, theory-guided interpretation of formal institutional documents — statutes, bylaws, regulations, manifestos, policy papers, minutes, organisational reports, and digital-native sources such as party platform interfaces and archived webpages. In this approach, Qualitative Content Analysis provides the principal analytical framework through which documentary materials are systematically coded and interpreted.

Advantages:

• Allows direct access to formal institutional rules as expressed by parties themselves. Non-reactive: documents exist independently of the research process.
• Enables comparative analysis across parties, countries, and time periods. Combines inductive exploration with deductive theory-testing.
• Produces transparent and systematically documented coding schemes.

Disadvantages

• Cross-national comparative designs face additional challenges of conceptual equivalence, legal translation, and differences in national statutory traditions, which require explicit methodological reflection.
• Documents reflect formal rules, not necessarily practice; gap between statute and implementation.
• Availability and accessibility of documents vary across parties and countries; availability bias may affect corpus composition.

Documents may be incomplete, outdated, or available only in languages requiring translation, introducing additional interpretive challenges. Differences between public statutes and internal regulations may not always be resolvable from available sources. Digital-native sources (party websites, platform interfaces, app content) change frequently and may not persist online; researchers using such materials should capture and archive dated versions (e.g. via the Wayback Machine or timestamped exports) so the corpus remains auditable over time.

Where internal or restricted-access documents are used, researchers should consider the ethical implications of obtaining and citing materials not intended for public circulation, including questions of consent, confidentiality, and potential repercussions for sources within the organisation. Interpretation requires linguistic and political context knowledge.

Inter-coder reliability can be challenging to achieve in purely qualitative designs.

Define research question and identify relevant document types (e.g., party statutes, internal regulations, manifestos, policy programmes, minutes, digital governance documents).

Build a corpus: collect documents systematically, recording version, date, official source, and language; state explicit inclusion and exclusion criteria (e.g. type of organisation covered, time period, language restrictions, and how missing or untranslated documents were handled), since corpus boundaries directly affect what the analysis can and cannot claim.

Develop a coding framework: this approach can be conducted deductively through theory-driven coding categories, inductively through category development from the documents themselves, or through iterative combinations of both. Categories should be grounded in the theoretical framework established in step 1.

Pilot coding: apply framework to a subset of documents; refine categories.

Full coding: apply categories systematically across the corpus. Qualitative data analysis software (ATLAS.ti, MAXQDA, NVivo) can support this process, but structured coding matrices in spreadsheet format are a valid alternative for smaller corpora; for scanned or image-based archival documents, OCR tools (e.g. ABBYY FineReader, Adobe Acrobat, or the open-source Tesseract) should be used to produce machine-readable text before coding.

Assess reliability: where multiple coders independently apply a shared coding scheme — the typical situation in larger team-based projects — calculate inter-coder reliability on a random subset (≥ 20%) using Krippendorff’s alpha, which is suitable for multiple coders and different levels of measurement; values around 0.80 are commonly considered indicative of good reliability (Krippendorff 2004), although acceptable thresholds vary depending on the level of interpretive inference required.

In single-coder or strongly interpretive designs, where categories require sustained contextual judgment, reflexive consistency and documented transparency of coding decisions are the more relevant quality criterion, and this choice should be stated explicitly. Interpret findings in theoretical context, triangulating systematically with other sources where possible — for instance, cross-checking a statutory provision against media coverage, member accounts, or direct observation of the platform it describes, which is how the gap between formal rule and actual practice (noted above as a key limitation) can be empirically assessed rather than assumed.

Report: document corpus composition and selection criteria, coding framework, reliability scores, and key quotations as evidence. Scott’s (1990) four documentary quality criteria — authenticity, credibility, representativeness, and meaning — should guide both corpus construction and reporting.

Applied example: in a statutory analysis of 20 European parties across seven countries (2022–2025), coding categories included digital infrastructure control, contestation mechanisms, offline channel guarantee, and deliberative override capacity.

Key quality standards:

Transparency: full documentation of corpus, coding procedure, and decision rules. Reliability: inter-coder reliability assessment is appropriate in team-based coding projects. Krippendorff’s alpha is recommended as the primary reliability measure, with values around 0.80 commonly indicating good reliability (Krippendorff 2004), although acceptable levels depend on research objectives, coding complexity, and methodological tradition.

In interpretive qualitative designs, reflexive consistency and transparency of coding decisions may be more relevant quality criteria than statistical agreement measures. These two criteria are not interchangeable, and a study should state explicitly which applies to its design: Krippendorff’s alpha is expected whenever multiple coders independently apply a shared, relatively determinate coding scheme to the same corpus — the standard situation in larger team-based comparative projects; reflexive consistency and documented interpretive judgment become the primary quality criterion in single-coder or strongly interpretive designs, where the categories themselves require sustained contextual judgment that a simple agreement statistic would not adequately capture.

Validity: coding categories must be grounded in theory and piloted against the material. Category construction and coding procedures should follow established qualitative content analysis guidelines (Mayring 2015; Schreier 2012; Kuckartz 2014). Documentary quality (Scott 1990): documents should be assessed for four criteria — authenticity (is this the genuine original?), credibility (is the content free from distortion?), representativeness (is the corpus complete or systematically bounded?), and meaning (is the content clear and interpretable in context?).

Reflexivity: researcher position and interpretive decisions should be acknowledged throughout the analytical process.

Transferability: sufficient contextual information should be provided to allow researchers to assess whether findings may apply to comparable cases. Transparency of documentation should allow other researchers to audit, understand, and potentially replicate the coding procedure. In layman’s terms: Did different coders reading the same document reach the same conclusions? If yes, the coding framework is reliable. Do the categories actually capture what the theory says matters? If yes, it is valid. Is the document what it claims to be, and does the corpus cover what it should? These are the documentary quality questions.

Bowen, G. A. (2009). Document analysis as a qualitative research method. Qualitative Research Journal, 9(2), 27–40.

Katz, R. S. & Mair, P. (eds.) (1992). Party Organizations: A Data Handbook on Party Organizations in Western Democracies, 1960–90. London: Sage.

Kim, D. S. (2022). Taming abundance: Doing digital archival research (as political scientists). PS: Political Science & Politics, 55(3), 530–538.

Krippendorff, K. (2004). Content Analysis: An Introduction to Its Methodology (2nd ed.). Thousand Oaks, CA: Sage. Kuckartz, U. (2014). Qualitative Text Analysis: A Guide to Methods, Practice and Using Software. London: Sage.

Landis, J. R. & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.

Mayring, P. (2015). Qualitative Inhaltsanalyse. Grundlagen und Techniken (12th ed.). Weinheim/Basel: Beltz.

Schreier, M. (2012). Qualitative Content Analysis in Practice. London: Sage. Scott, J. (1990). A Matter of Record: Documentary Sources in Social Research. Cambridge: Polity Press.

Duverger, M. (1954). Political Parties: Their Organization and Activity in the Modern State. John Wiley & Sons — foundational comparative documentary analysis of party statutes and organisational rules across European democracies.

Katz, R. S. & Mair, P. (eds.) (1992). Party Organizations: A Data Handbook on Party Organizations in Western Democracies, 1960–90. London: Sage—landmark systematic comparative documentary analysis of party rules across 79 parties.

Gauja, A. (2013). Party Reform: The Causes, Challenges and Consequences of Organisational Change. Oxford University Press — comparative documentary analysis of party statutes and reform processes across established democracies.

Sánchez Medero, G. (2026). Replication data: Digitalisation without democratisation (DOC framework). Zenodo. DOI: 10.5281/zenodo.20584273—application of the method to statutory analysis of 20 European parties across seven countries, including full codebook and coded dataset.

Poguntke, T., Scarrow, S. E., Webb, P. D. et al. (2016). ‘Party Rules, Party Resources and the Politics of Parliamentary Democracies: How Parties Organize in the 21st Century.’ Party Politics, 22(6), 661–678.

Pacześniak, A. (2026). Digitalization of Political Parties in Poland: Between Law and Practice. Bialystok Legal Studies, 31(1), 273–288 — recent qualitative content analysis of seven Polish parties’ statutes, websites, and digital platforms, using the DIGIPART operationalization to assess internal and external dimensions of party digitalization.

Example data sets

• Replication dataset (Sánchez Medero, UCM): coded statutory corpus, codebook, and replication materials for the analysis (deliberative override capacity), covering 20 European parties across seven countries (2022–2025). Available via Zenodo: https://zenodo.org/records/20584273 (DOI: 10.5281/zenodo.20584273; CC BY 4.0).
• MARPOR / Manifesto Corpus: collection of party electoral programmes with content-analytical coding, covering parties from over 60 countries since 1945. Available at: https://manifesto-project.wzb.eu 
• Political Party Database (PPDB): comparative data on party rules, resources, and internal decision-making across 51 countries (Round 2); includes a party statute archive directly relevant for documentary analysis. Available at: https://www.politicalpartydb.org

• Anika Gauja, University of Sydney
• Thomas Poguntke, Heinrich-Heine-Universität Düsseldorf, emeritus
• Gema Sánchez Medero, Universidad Complutense de Madrid, Spain.
• Susan E. Scarrow, University of Houston

What a journalist or civic-education instructor most needs to understand about documentary analysis is that it examines what an organisation formally commits to on paper, not what it necessarily does in practice. Reading a party’s statutes or internal regulations is similar to reading the bylaws of a club or association: the document tells you what members are formally entitled to, who holds decision-making power, and what procedures must be followed — but it does not tell you whether those rules are actually followed, contested, or quietly bypassed in day-to-day organisational life. This is exactly the value of the method for reporting on party digitalization: a party’s statutes might formally guarantee members a binding vote on policy through a digital platform, while in practice that platform is rarely used, easily overridden by leadership, or technically difficult to access. Comparing the formal rule against other evidence — news coverage, member accounts, observed platform behaviour — is what reveals this gap, and documentary analysis is the systematic way of establishing precisely what the formal rule says before making that comparison. What the method cannot do on its own is equally important: a statute search does not replace interviews with members or staff, does not capture informal practice, and does not establish that a documented rule is actually enforced. Saying “the party’s statutes formally guarantee X” is an accurate, verifiable claim from this method; saying “the party actually does X” requires additional evidence beyond the documents themselves.