4 Summarizing Simulation Results
4.1 Tables and Figures
For plotting simulation results:
For making tables with simulation results:
- Check out R Markdown
- RStudio guidelines for
R
Markdown: https://rmarkdown.rstudio.com/ R
Markdown: The Definitive Guide: https://bookdown.org/yihui/rmarkdown/- Reproducible APA manuscripts with
R
Markdown: https://crsh.github.io/papaja_man/reporting.html - HTML tables with
knitr::kable
andkableExtra
: http://haozhu233.github.io/kableExtra/ - A nice blog post about stylish tables in
R
: https://www.littlemissdata.com/blog/prettytables
- RStudio guidelines for
4.2 Exporting the Results
I recommend exporting all the simulation results if:
- writing all the results into a file would not take a lot of space in the computer
- existing computations are heavy and thus it is hard to summarize the results right away
- it is likely that the simulation may be interrupted for some reason
Regardless what we decide to save (either all output or the summarized output), a nice way to save the results is to place a write
function inside the loop. Using the nested foreach
loops as an example, we will first create an empty .csv file called “results.csv”:
write.table(matrix(c("nitem", "nexaminee", "guess", "parameter", "bias", "rmse", "correlation"),
nrow = 1,
ncol = 7),
file = "results.csv",
sep = ",",
col.names = FALSE,
row.names = FALSE)
Then, we modify our code in a way that we will not save the results and instead write them into the results.csv file.
# Set all the conditions
iterations = 100
seed = sample.int(10000, 100)
nitem = c(10, 15, 20, 25)
nexaminee = c(250, 500, 750, 1000)
guess = c(-1, 0.16)
# Register four clusters
cl <- makeCluster(4)
registerDoParallel(cl)
# Run nested foreach loops
foreach(i=1:iterations,
.packages = c("mirt", "doParallel", "dplyr"),
.combine = rbind) %:%
foreach(j=nitem,
.packages = c("mirt", "doParallel", "dplyr"),
.combine = rbind) %:%
foreach(k=nexaminee,
.packages = c("mirt", "doParallel", "dplyr"),
.combine = rbind) %:%
foreach(m=guess,
.packages = c("mirt", "doParallel", "dplyr"),
.combine = rbind) %dopar% {
# Generate item parameters and data
step1 <- generate_data(nitem=j, nexaminee=k, seed=seed[i])
# Estimate item parameters
step2 <- estimate_par(step1$respdata, guess = m)
# Summarize results
step3 <- summarize(step2, step1$itempar)
# Finalize results
final <- step3 %>%
group_by(parameter) %>%
summarise(bias = round(mean(bias),3),
rmse = round(mean(rmse),3),
correlation = round(mean(correlation),3)) %>%
mutate(nitem = j,
nexaminee = k,
guess = m) %>%
select(nitem, nexaminee, guess, parameter, bias, rmse, correlation) %>%
as.data.frame()
# Write the results
write.table(final, "results.csv",
sep = ",",
col.names = FALSE,
row.names = FALSE,
append = TRUE) # This will keep the file open for appending the results
}
# Stop the clusters
stopCluster(cl)
References
Sarkar, Deepayan. 2018. Lattice: Trellis Graphics for R. https://CRAN.R-project.org/package=lattice.
Sievert, Carson, Chris Parmer, Toby Hocking, Scott Chamberlain, Karthik Ram, Marianne Corvellec, and Pedro Despouy. 2020. Plotly: Create Interactive Web Graphics via ’Plotly.js’. https://CRAN.R-project.org/package=plotly.
Wickham, Hadley, Winston Chang, Lionel Henry, Thomas Lin Pedersen, Kohske Takahashi, Claus Wilke, Kara Woo, and Hiroaki Yutani. 2019. Ggplot2: Create Elegant Data Visualisations Using the Grammar of Graphics. https://CRAN.R-project.org/package=ggplot2.