# Basic usage

This vignette describes the most basic usage of the `sentopics` package by estimating an LDA model and analysis it’s output. Two other vignettes, describing time series and topic models with sentiment are also available.

# Data

The package is shipped with a sample of press conferences from the European Central bank. For ease of use, the press conferences have been pre-processed into a `tokens` object from the `quanteda` package. (See quanteda’s introduction for details on these objects). The press conferences also contains meta-data which can be accessed using `docvars()`.

The press conferences were obtained from ECB’s website. The package also provides an helper function to replicate the creation of the dataset: `get_ECB_press_conferences()`

    library("sentopics")
    data("ECB_press_conferences_tokens")
    print(ECB_press_conferences_tokens, 3)
    # Tokens consisting of 3,860 documents and 5 docvars.
    # 1_1 :
    #  [1] "outcome"           "meeting"           "decision"
    #  [4] ""                  "ecb"               "general"
    #  [7] "council"           "governing_council" "executive"
    # [10] "board"             "accordance"        "escb"
    # [ ... and 7 more ]
    #
    # 1_2 :
    #  [1] ""              "state"         "government"    "member"
    #  [5] "executive"     "board"         "ecb"           "president"
    #  [9] "vice"          "president"     "date"          "establishment"
    # [ ... and 13 more ]
    #
    # 1_3 :
    #  [1] ""          "meeting"   "executive" "board"     "meeting"   ""
    #  [7] "general"   ""          "meeting"   ""
    #
    # [ reached max_ndoc ... 3,857 more documents ]
    head(docvars(ECB_press_conferences_tokens))
    #        .date doc_id                                        title
    # 1 1998-06-09      1 ECB Press conference: Introductory statement
    # 2 1998-06-09      1 ECB Press conference: Introductory statement
    # 3 1998-06-09      1 ECB Press conference: Introductory statement
    # 4 1998-06-09      1 ECB Press conference: Introductory statement
    # 5 1998-06-09      1 ECB Press conference: Introductory statement
    # 6 1998-06-09      1 ECB Press conference: Introductory statement
    #                                                               section_title
    # 1 Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    # 2 Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    # 3 Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    # 4 Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    # 5 Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    # 6 Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    #    .sentiment
    # 1 -0.01470588
    # 2 -0.02500000
    # 3  0.00000000
    # 4  0.00000000
    # 5  0.00000000
    # 6  0.00000000

# Topic modeling

## Introduction

`sentopics` implements three types of topic model. The simplest, Latent Dirichlet Allocation (LDA), assumes that textual documents are issued from a generative process involving \(K\) topics.

A given document \(d\) is constituted of a list of words \(d = (w_1, \dots, w_N)\), with \(N\) being the document’s length. Each word \(w_i\) originates from a vocabulary consisting of \(V\) distinct terms. Then, documents are generated from the following random process:

1. For each topic \(k \in K\), a distribution \(\phi_k\) over the vocabulary is drawn. This distribution represent the probability of a word appearing given it belong to the topic and is drawn from a Dirichlet distribution with hyperparameter \(\beta\). \[\phi \sim Dirichlet(\beta)\]
2. For each document, a mixture of the \(K\) topics, \(\theta_d\), assign the probability of a word in document \(d\) being generated from topic \(k\). This mixture is also drawn from a Dirichlet distribution with hyperparameter \(\alpha\). \[\theta \sim Dirichlet(\alpha)\]
3. For each word position \(i\) of document \(d\), the following sequence of draws is executed:
   1. A latent topic assignment \(z_i\) is drawn from the document mixture. \(z_i \sim Multinomial(\theta)\)
   2. A word \(w_i\) is drawn from the topic’s vocabulary distribution. \(w_i \sim Multinomial(\phi_{z_i})\)

In `sentopics` the LDA model is estimated through Gibbs sampling, that iteratively sample the topic assignment \(z_i\) of every word of the corpus until reaching a convergence. The topic assignments are sampled from the following distribution: \[ p(z_i = k|w,z^{-i}) \propto \frac{n_{k,v,.}^{-i} + \beta}{n_{k,.,.}^{-i} + V\beta} \frac{n_{k,.,d}^{-i} + \alpha}{n_{.,.,d}^{-i} + K\alpha},\] where \(n_{k,v,d}\) is the count of words at index \(v\) of the vocabulary, assigned to topic \(k\) and part of document \(d\). The replacement of one of the indices \(\{k,v,d\}\) by a dot indicates instead the count for all topics, all vocabulary indices or all documents. The superscript \(-i\) indicates that the current word position \(i\) is left out from the count variables.

## Estimating LDA models with `sentopics`

The estimation of an LDA model is easily replicated using the `LDA()` and `grow()` function. The first function prepares the `R` object and initialize the assignment of the latent topics. The second function estimates the model using Gibbs sampling for a given number of iterations. Note that `grow()` may be used to iterate the model multiple times without resetting the estimation.

    set.seed(123)
    lda <- LDA(ECB_press_conferences_tokens)
    lda
    # An LDA model with 5 topics. Currently grown by 0 Gibbs sampling iterations.
    # ------------------Useful methods------------------
    # grow      :Iterates the model using Gibbs sampling
    # topics    :Return the most important topic of each document
    # topWords  :Return a data.table with the top words of each topic/sentiment
    # plot      :A sunburst chart representing the estimated mixtures
    # This helpful message is displayed once per session, unless calling `print(x, extended = TRUE)`
    lda <- grow(lda, iterations = 100)
    lda
    # An LDA model with 5 topics. Currently grown by 100 Gibbs sampling iterations.

Internally, the `lda` object is stored as a list and contains the model’s parameters and outputs.

    str(lda, max.level = 1, give.attr = FALSE)
    # List of 10
    #  $ tokens       :List of 3860
    #  $ vocabulary   :Classes 'data.table' and 'data.frame':   1168 obs. of  3 variables:
    #  $ K            : num 5
    #  $ alpha        : num [1:5, 1] 1 1 1 1 1
    #  $ beta         : num [1:5, 1:1168] 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 0.01 ...
    #  $ it           : num 100
    #  $ za           :List of 3860
    #  $ theta        : num [1:3860, 1:5] 0.0455 0.0357 0.0909 0.0667 0.0625 ...
    #  $ phi          : num [1:1168, 1:5] 4.32e-07 4.32e-07 4.32e-07 6.47e-03 4.32e-07 ...
    #  $ logLikelihood: num [1:100, 1] -943778 -927554 -912671 -893733 -864875 ...

`tokens` is the initial tokens object used to create the model. `vocabulary` is a data.frame indexing the set of words. `K` is the number of topics. `alpha` is the hyperparameter of the document-topic mixtures. `beta` is the hyperparameter of the topic-word mixtures. `it` is the number of iterations of the model. `za` contains the topic assignments of each word of the corpus. `theta` are the estimated document-topic mixtures. `phi` are the estimated topic-word mixtures. `logLikelihood` is the log-likelihood of the model at each iteration.

Estimated mixtures are easily accessible through the `$` operator. But the package also includes the `topWords()` function to extract the most probable words of each topic. `topWords()` includes three types of outputs: long `data.table`/`data-frame`, `matrix` or `ggplot` object (also accessible through the alias `plot_topWords()`).

    head(lda$theta)
    #       topic
    # doc_id     topic1     topic2    topic3     topic4     topic5
    #    1_1 0.04545455 0.04545455 0.7727273 0.04545455 0.09090909
    #    1_2 0.03571429 0.14285714 0.7500000 0.03571429 0.03571429
    #    1_3 0.09090909 0.09090909 0.6363636 0.09090909 0.09090909
    #    1_4 0.06666667 0.06666667 0.7333333 0.06666667 0.06666667
    #    1_5 0.06250000 0.06250000 0.7500000 0.06250000 0.06250000
    #    1_6 0.05263158 0.10526316 0.5789474 0.10526316 0.15789474
    topWords(lda, output = "matrix")
    #       topic1           topic2       topic3              topic4
    #  [1,] "price"          "fiscal"     "governing_council" "growth"
    #  [2,] "inflation"      "euro_area"  "ecb"               "quarter"
    #  [3,] "development"    "growth"     "meeting"           "loan"
    #  [4,] "annual"         "country"    "president"         "financial"
    #  [5,] "increase"       "policy"     "bank"              "euro_area"
    #  [6,] "projection"     "reform"     "operation"         "rate"
    #  [7,] "hicp"           "structural" "outcome"           "sector"
    #  [8,] "oil"            "market"     "press"             "condition"
    #  [9,] "euro_area"      "economic"   "vice"              "annual"
    # [10,] "inflation_rate" "measure"    "euro"              "credit"
    #       topic5
    #  [1,] "risk"
    #  [2,] "economic"
    #  [3,] "monetary"
    #  [4,] "price_stability"
    #  [5,] "euro_area"
    #  [6,] "development"
    #  [7,] "interest_rate"
    #  [8,] "outlook"
    #  [9,] "monetary_policy"
    # [10,] "growth"

In addition, document-level is facilitated through the use of the `melt()` method, that joins estimated topical proportions to document metadata present in the `tokens` input. This result in a long `data.table`/`data.frame` that can be used for plotting or easily reshaped to a wide format (for example using `data.table::dcast`).

    melt(lda, include_docvars = TRUE)
    #         topic       prob      .date    .id doc_id
    #     1: topic1 0.04545455 1998-06-09    1_1      1
    #     2: topic1 0.03571429 1998-06-09    1_2      1
    #     3: topic1 0.09090909 1998-06-09    1_3      1
    #     4: topic1 0.06666667 1998-06-09    1_4      1
    #     5: topic1 0.06250000 1998-06-09    1_5      1
    #    ---
    # 19296: topic5 0.28947368 2021-12-16 260_20    260
    # 19297: topic5 0.10526316 2021-12-16 260_21    260
    # 19298: topic5 0.05000000 2021-12-16 260_22    260
    # 19299: topic5 0.41025641 2021-12-16 260_23    260
    # 19300: topic5 0.14285714 2021-12-16 260_24    260
    #                                               title
    #     1: ECB Press conference: Introductory statement
    #     2: ECB Press conference: Introductory statement
    #     3: ECB Press conference: Introductory statement
    #     4: ECB Press conference: Introductory statement
    #     5: ECB Press conference: Introductory statement
    #    ---
    # 19296:                             PRESS CONFERENCE
    # 19297:                             PRESS CONFERENCE
    # 19298:                             PRESS CONFERENCE
    # 19299:                             PRESS CONFERENCE
    # 19300:                             PRESS CONFERENCE
    #                                                                             section_title
    #     1:          Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    #     2:          Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    #     3:          Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    #     4:          Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    #     5:          Willem F. Duisenberg, President of the European Central Bank, 9 June 1998
    #    ---
    # 19296: Christine Lagarde, President of the ECB,Luis de Guindos, Vice-President of the ECB
    # 19297: Christine Lagarde, President of the ECB,Luis de Guindos, Vice-President of the ECB
    # 19298: Christine Lagarde, President of the ECB,Luis de Guindos, Vice-President of the ECB
    # 19299: Christine Lagarde, President of the ECB,Luis de Guindos, Vice-President of the ECB
    # 19300: Christine Lagarde, President of the ECB,Luis de Guindos, Vice-President of the ECB
    #         .sentiment
    #     1: -0.01470588
    #     2: -0.02500000
    #     3:  0.00000000
    #     4:  0.00000000
    #     5:  0.00000000
    #    ---
    # 19296: -0.01960784
    # 19297:  0.00000000
    # 19298:  0.05555556
    # 19299: -0.01052632
    # 19300:  0.00000000

To ease the result analysis, we can rename the default topic labels using the `sentopics_labels()` function. As a result, all outputs of the model will now display the custom labels.

    sentopics_labels(lda) <- list(
      topic = c("Inflation", "Fiscal policy", "Governing council", "Financial sector", "Uncertainty")
    )
    head(lda$theta)
    #       topic
    # doc_id  Inflation Fiscal policy Governing council Financial sector Uncertainty
    #    1_1 0.04545455    0.04545455         0.7727273       0.04545455  0.09090909
    #    1_2 0.03571429    0.14285714         0.7500000       0.03571429  0.03571429
    #    1_3 0.09090909    0.09090909         0.6363636       0.09090909  0.09090909
    #    1_4 0.06666667    0.06666667         0.7333333       0.06666667  0.06666667
    #    1_5 0.06250000    0.06250000         0.7500000       0.06250000  0.06250000
    #    1_6 0.05263158    0.10526316         0.5789474       0.10526316  0.15789474
    plot_topWords(lda) + ggplot2::theme_grey(base_size = 9)

Besides modifying topic labels, it is also possible to merge topics into a greater thematic. This is often useful when estimating a large number of topics (e.g, K > 15). The `mergeTopics()` does this job and re-label topics accordingly.

    merged <- mergeTopics(lda, list(
      `Big big thematic` = c(1, 3:5),
      `Fical policy` = 2
    ))
    merged
    # An LDA model with 2 topics. Currently grown by 100 Gibbs sampling iterations.

Note that merging topics is only useful for presentation purpose. Using again `grow` on a model with merged topics will drastically change the results as the current state of the model does not results from a standard estimation with the merged set of parameters.

Provided that the `plotly` package is installed, one can also directly use `plot()` on the estimated topic model to enjoy a dynamic view of topic proportions and their most probable words (presented as a screenshot hereafter to limit this vignette’s size).

    plot(lda)