This week I had conducted my last pilot for the study on Monday. During the pilot we had discovered a bug with the a new feature that I had implemented on the platform, therefore I spent the first half of the week fixing the bug on the platform. During the first half of the week, I also worked on finishing up a first draft of my final report for the DREU program. After my the first draft of my final report, I had gotten some feedback from Dr. Battle regarding a couple of section in the report. I have submitted all other parts of the fourth milestone, and I have been working on feedback that I recieved on the report. The only items that remain are handing in the finished final report and the integrating the new demo video of the platform after the bug was fixed.

This week I finished the final edits the experiment platform (surveys and task) after a couple of revisions. I also recruited for and conducted all most all of the pilots for our experiment this week. The pilots were insightful and provided a good perspective of the bugs that remain in the system, survey errors, and overall feedback on the study workflow and process. The most surprising issue that arose during the pilot was a lagging issue with one pilot participant that was in the US; this made us realize that fast enough internet connection is vital for the experiment to be conducted due to the hefty processing required to render the visualizations in the Voyager platform.

This week I finished up most of the experiment platform and hosted the study publicly through UMD’s UMIACS department. The challenge with finishing the platform for the qualtrics section was working on the randomization aspect, and for the task section was storing logs and publicly hosting the website on UMD’s server due to the amount of steps and permissions required. Since I also stared to recruit for the pilots this week, I also needed to make sure all the accompanying documents to run the study were all set; the documents inculded a video session script and several data collection organization documents.

This week I finished the revisions for the IRB, and it was approved by mid-week! We changed up the experiment protocol up a bit again in terms of administering the creativity pre- and post- task surveys. We decided to increase the time limit to 3 minutes and people being able to list a total of five words that they associate with that shape or color. We also played around with the idea of having people list as many words as possible, however we realized the analysis associated with that would be extremely messy and possibly affect what we are directly measuring for, so we opted for more control. Another struggle during the week was trying to figure out a way to analyze the creativity surveys and how to account for words that were not already mentioned in our association list and if the participant did not list five words. I also began reading the Using Metrics of Curation to Evaluate Information-Based Ideation by Kerne et al. which involves information-based ideation through Pinterest boards which is the approach that we plan on using in our experiment.

This week I finished reading the Bohemian Bookshelf paper and wrote a short summary regarding the paper, and I also completed the IRB feedback edits that I received (along with our new protocol edits). We received some great feedback from Dr. Remco Chang and decided to change up our experiment protocol again and incorporate several more surveys to the experiment, as we decided to deem some of the protocol necessary for what we’re trying to measure. These edits allowed us to measure more variables in our experiment.

This week I made a couple of edits to the pre-registration’s analysis section that we plan on submitting to aspredicted.org, and I spent most of my time working on the platform. We received some great feedback from Dr. Michael Correll and decided to change up our experiment protocol and shorten the experiment a bit, as we decided to deem some of the protocol unnecessary for what were trying to measure. These edits allowed for the experiment platform to be extremely simplified.

During the week, I worked on finishing the write up of the experiment protocol so that we were able to send it out to a couple of professors at the department for a review/sanity check. I also conducted a literature search and found a few relevant articles that I put on our list of papers to read and possibly cite. Conducting the literature search and iterating on the experiment protocol led to me discovering that we may need to change our scoring analysis of creativity while conducting the study. I found two different word association norms/thesauruses that are relevant to our study and that have been used before, however both of them are old (from the 1970s), and there are some downfalls to both that I found. This search lead to us changing our analysis approach a little bit. I also started to work on integrating Voyager into the codebase and use another lab member’s previous work, however is it becoming evident that I most likely need to create my platform from scratch. The Voyager codebase is also mainly in TypeScript so it is definitely something that I need to pick up.

This week I completed the last review of the IRB documents and turned them in to the review board. I also finished the pre-registration after an iteration, got caught up on the paper summaries in our article’s related work section, and selected the datasets that we will be using for the experiment. In order to begin work on the experiment platform, I set up a meeting with one of the other students in the lab to discuss a codebase that they had created for another experiment previously. I got good insights, and plan on using that codebase as a starting point for building this study’s platform.

This week my main focus was finalizing the IRB paper work and starting the experiment protocol and pre-registration for the project. I was able to have multiple folks in the lab review my work, which provided a bunch of insights into how to improve my writing surrounding this topic. During this process, I was able to hash out a greater number of lower-level details about the exact experiment protocol that we will be using during the process. This insight allowed me to create a detailed timeline for the rest of my time at DREU and the experiment, in a general sense. I was also able to begin setting up my environment, and mapping out the experiment platform that I will need to build starting next week to conduct the study. I realized through the mapping that I would need to build the platform a bit out of order, and will possibly use another study’s code base (in my lab) as a base to build off of, as one of the 3 tools that we’re using is already integrated and customized in that code base.

This week my main focus was setting up my environment (Overleaf, GitHub, Server, etc.), and continuing to write the IRB supplemental documents to send them for review. I had a starting meeting with my mentor, Dr. Battle, to better outline my goals and weekly tasks. We decided on having 2 meetings a week (Tuesday/Thursday), while my weekly tasks (in addition to my longer-term goals) will include conducting a literature search, reading 2 articles a week and summarizing them, and writing/iterating on a section of the final article of this study. I learned a bit more about how the experiment will be conducted, and we finalized on the 3 tools and surveys that we will be using for the experiment (Voyager, Jupyter, and Google Sheets). I’m excited to learn a bit more about Voyager, as it is a tool that I’m not particularly as familiar with compared to the rest of the tools we will be using. During this week, I also go a chance to get a better look at some of the other papers from the lab like the lab’s Vis Ex-Machina paper.