4 Summarizing Simulation Results

4.1 Tables and Figures

For plotting simulation results:

• ggplot2 (Wickham, Chang, et al. 2019)
• lattice (Sarkar 2018)
• plotly (Sievert et al. 2020)

For making tables with simulation results:

• Check out R Markdown
  • RStudio guidelines for R Markdown: https://rmarkdown.rstudio.com/
  • R Markdown: The Definitive Guide: https://bookdown.org/yihui/rmarkdown/
  • Reproducible APA manuscripts with R Markdown: https://crsh.github.io/papaja_man/reporting.html
  • HTML tables with knitr::kable and kableExtra: http://haozhu233.github.io/kableExtra/
  • A nice blog post about stylish tables in R: https://www.littlemissdata.com/blog/prettytables

4.2 Exporting the Results

I recommend exporting all the simulation results if:

• writing all the results into a file would not take a lot of space in the computer
• existing computations are heavy and thus it is hard to summarize the results right away
• it is likely that the simulation may be interrupted for some reason

Regardless what we decide to save (either all output or the summarized output), a nice way to save the results is to place a write function inside the loop. Using the nested foreach loops as an example, we will first create an empty .csv file called “results.csv”:

write.table(matrix(c("nitem", "nexaminee", "guess", "parameter", "bias", "rmse", "correlation"), 
                   nrow = 1, 
                   ncol = 7), 
            file = "results.csv", 
            sep = ",",
            col.names = FALSE, 
            row.names = FALSE)

Then, we modify our code in a way that we will not save the results and instead write them into the results.csv file.

# Set all the conditions
iterations = 100
seed = sample.int(10000, 100)
nitem = c(10, 15, 20, 25)
nexaminee = c(250, 500, 750, 1000)
guess = c(-1, 0.16)

# Register four clusters
cl <- makeCluster(4) 
registerDoParallel(cl)

# Run nested foreach loops
foreach(i=1:iterations, 
        .packages = c("mirt", "doParallel", "dplyr"),
        .combine = rbind) %:%
  
  foreach(j=nitem, 
          .packages = c("mirt", "doParallel", "dplyr"),
          .combine = rbind)  %:%
  
 foreach(k=nexaminee, 
          .packages = c("mirt", "doParallel", "dplyr"),
          .combine = rbind)  %:%
  
  foreach(m=guess, 
          .packages = c("mirt", "doParallel", "dplyr"),
          .combine = rbind)  %dopar% {
            # Generate item parameters and data
            step1 <- generate_data(nitem=j, nexaminee=k, seed=seed[i])
            # Estimate item parameters
            step2 <- estimate_par(step1$respdata, guess = m)
            # Summarize results
            step3 <- summarize(step2, step1$itempar)
            # Finalize results
            final <- step3 %>%
              group_by(parameter) %>%
              summarise(bias = round(mean(bias),3),
                        rmse = round(mean(rmse),3),
                        correlation = round(mean(correlation),3)) %>%
              mutate(nitem = j,
                     nexaminee = k,
                     guess = m) %>%
              select(nitem, nexaminee, guess, parameter, bias, rmse, correlation) %>%
              as.data.frame()
            # Write the results
            write.table(final, "results.csv", 
                        sep = ",", 
                        col.names = FALSE, 
                        row.names = FALSE, 
                        append = TRUE) # This will keep the file open for appending the results
            }

# Stop the clusters
stopCluster(cl)

References

Sarkar, Deepayan. 2018. Lattice: Trellis Graphics for R. https://CRAN.R-project.org/package=lattice.

Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2020. Plotly: Create Interactive Web Graphics via ’Plotly.js’. https://CRAN.R-project.org/package=plotly.

Wickham, Hadley, Winston Chang, Lionel Henry, Thomas Lin Pedersen, Kohske Takahashi, Claus Wilke, Kara Woo, and Hiroaki Yutani. 2019. Ggplot2: Create Elegant Data Visualisations Using the Grammar of Graphics. https://CRAN.R-project.org/package=ggplot2.