Michael Smith writes:
I have a research challenge and I was hoping you could spare a minute of your time. I hope it isn’t a bother—I first came across you when I saw your post on how psychology researchers can learn from statisticians. I figure even if you don’t know the answer to this question, you might know someone who would. My colleagues and I want to explore implicit biases using the trolley problem as the mechanism for discovering these biases. The problem we have is we have very specific needs for our survey software. I don’t know enough about what is out there to make an informed decision. What we need is a salient timer on the screen, counting down from 5 seconds. There will be two images, one on the left and one on the right. In one condition, when those 5 seconds are up, we want the software to read “Dead” on one of those images, the default being the people in the image on the right will “die”. In the other condition, we want the software to read “Saved” on one of those images. I’m not sure we can have the effect we want without that added touch. Thanks in advance for any ideas you have!
My reply: I have no idea, but I’m sure this is possible. Maybe one of our readers has a suggestion?
Maybe create a gif with the timer + images and then just find a software that can show an image along with a question.
Cheapest way out is to hire a student who can do basic web programming. Or hire a freelance web dev. Survey companies will be happy to implement custom surveys for top $.
Could definitely be accomplished with LimeSurvey (http://www.limesurvey.org) and an animated gif (as K suggests), or JavaScript. This would allow you to avoid having to create an entire survey engine backend (or pay to have one created). If you’re interested in this option and need assistance in implementation, let me know. Contact info is on my website.
(Andrew: if you’re uncomfortable with this kind of self-promotion/networking, feel free to delete the last two sentences.)
Software implementation aside, the basic premise makes me somewhat skeptical.
Do people really map a gif from a computer screen showing “dead” vs “saved” with high fidelity to whatever are their biases in the real world? Isn’t this like screening for good Marines by a first person shooter PC game?
You should direct him to The Experiment Factory, which is an open source infrastructure to develop experiments like this (expfactory.org and expfactory.github.io). If you post an issue on our experiments repo at (https://github.com/expfactory/expfactory-experiments) we can talk about how to make an experiment, and give support as well. I am a graduate student at Stanford, and the lead developer.
Free options are Ibexfarm (there’s not a built-in way to do this but if you know Javascript you can define an object to do it, I think) or DMDX (same, there’s not a built-in but I think you could hard-code something). If you have money to spend, Paradigm is very user-friendly and can be implemented online, but you need to buy a license.
(Follow-up: now that I’ve noticed K’s comment above, I actually think you could do this in Ibexfarm without complicated scripting… you can use Ibexfarm to display HTML, and if you combine that with K’s suggestion to make a GIF, it should be straightforward to just display an HTML page with that GIF embedded in it.)
This requires pretty basic coding skills. If you will often use computers in your research, it is worth learning to do this yourself. I can image what kind of other ineffeciencies abound in a lab where no one can easily figure out how to do this. Tons of time is probably being wasted on tasks that could be scripted.
Yeah, that’s a bit rude there.
It’s not meant to be rude, that was my own experience. I was fool. They’ll know if it applies to them when they think about all the hours spent entering/copying/pasting/formatting info, clicking around in excel, and attempting to place figures in the right place using word. Just spend the time upfront to learn basic coding skills.
I agree, people should learn to code — if it will help them with their research! So go learn R, for your analyses.
Coding survey software is a far different thing.
No one is saving any time by learning a web framework, learning how to make a timer interact with a question using flash or JS or whatevre, learning how to capture data into a database, validate it, export it for statistical analaysis. These are not trivial tasks to do well and “learning to code,” while nice, is going to take a lot of time.
This is like the definition of inefficiency: doing everything on your own by hand.
Scripting is great — so hire a programmer when needed. If your lab has lots of computing needs, time to hire someone for more than the odd job.
On a budget, the solution to me is to hire a competent software developer to deploy one of the open source packages discussed in this thread, battle test it a bit
While I mostly agree with what you are saying about coding a survey with the caveat that it is extremely beneficial to understand at least some coding for data extraction, transformation(cleaning), and loading(export). Having experience with a lot of different statistical software and data manipulation packages, I have yet to find a gui that allows one to create a data process comparable to that which can be done with code.
The gui’s tend to lack the flexibility, functionality and reproducibility of code for the same software and same projects. While there are often ‘workarounds’ for these, they tend to be far more difficult to troubleshoot than is code. Even for excel users, I would strongly recommend learning some VBscript to automate data processes.
If the data you are dealing with is simple and staightfoward and your analyses exploratory only, then stick with the gui. If it is not, then learn to code in the software package of your choice. You will benefit from the skill and learn to think about your data problems with greater insight. You do not need to become so skilled as to be a software developer, but you should be skilled at cleaning, transforming, and analyzing data with code.
I think we’re talking about different things here.
Cleaning data through a GUI is my idea of hell, so I’m with you. If you’re doing data analysis, someone on your team needs to be good with data manipulation using code (I use R, python is fine, if you like punishing yourself on complex data tasks, you might be able to get by with Stata. I hear people use SAS, and I guess if you’re good with MS tools those are fine too).
The OP, however, is about designing an interactive survey with special requirements. This involves far different skills than those of data cleaning, etc., even if those who are efficient with coding might be a quick study. You want something that will work when people use it, and the best way to assure that is to hire a pro who is good at these types of things. It’s not a complex project and someone will be able to do a good job.
I’m the first to tout that benefits of learning to code. But I’ve learned through time that when someone can do it better, and time is an issue, get help. This seems like one of those cases.
Agreed.
We had a Prof. in the Chemical Engineering Dept. who used to blow his own glassware. Admittedly, you can get these things done outside from craftsmen ( and I believe his lab did too) but then again there was no denying that the skill helped.
For one they never had to wait for a critical piece that failed. More importantly, since he know glassblowing, his apparatus used to be far more pragmatic and reliable since he know what could be done & what not and what sort of piece could take how much stress etc.
There’s some analogy here to designing your own web surveys. There’s no reason you *must*. But if you know what it takes, you can work with the process far better.
Programming it would certainly give you more flexibility, but you could also consider looking into using the built-in timers for online survey software from companies like the following:
SurveyGizmo:
Qualtrics:
It wouldn’t be too difficult to create different treatment conditions (randomly assigned based upon a random number generator and skip logic) and use the timer to advance the page or show/hide the image you want.
Sorry, the links didn’t post correctly (here’s the first one, http://help.surveygizmo.com/help/article/link/make-an-image-disappear-after-a-certain-amount-of-time; and here’s the second one, https://www.qualtrics.com/university/researchsuite/basic-building/editing-questions/question-types-guide/timing/)
I was also going to recommend Qualtrics — it’s not free, but it’s becoming increasingly popular with universities. You could do it with either the built in timer, or with added, custom javascript (should be “straightforward” to set up).
I use Adobe Flash to do this kind of thing. It gets knocked around as not being too modern, but the level of experimental control is really great. It does require a lot of hand coding, though.
Keep in mind you’re going to want to record responses — I’m assuming user inputs something at some point. Also maybe you want some demographic information, maybe IP address (crude measure to make sure people aren’t taking the survey multiple times), etc etc. So it’s not solely a gif, it’s a survey-based experiment. You may also want timing information, perhaps an attention check trial especially if you’re deploying it “in the wild,” treatment condition, maybe trial information, you need store all these resulting data. It’s good to have the researcher involved in what the output data will look like: this will be your dataset you analyze. Standard tools allow you to view, download, and monitor responses, so I’d lean more toward standard tools than writing everything yourself. Believe me, you probably will do some basic web programming even in a survey tool. But it takes care of a lot of stuff for you that is valuable.
Try http://www.psychopy.org/, this is becoming a popular psychophysics platform.
I was going to recommend psychopy (or Psychotoolbox), but at least according to Andrew’s title of the post, it is for an online survey.
Depending on what exactly you need and how urgently I might be able to knock something up for you. Send me a LinkedIn message if you are still looking for some help.
Dear OP,
I’d recommmend to you Tatool-web. You might need to code a bit for that (let me know if you need help), but I highly doubt you’ll find anything where you don’t need to write any line of code. And as a previous commenter said, if you’re doing research, coding is a must.
But are you sure you really want to do some trolleyology ? http://dx.doi.org/10.1016/j.cognition.2014.10.005
PS: Sorry for the late answer. I’m two weeks behind my RSS feed.