Temporal survey

Research
1

The Temporal proposal's champions group are trying to publicize the proposal and its corresponding polyfill in the developer community, and get feedback from people trying it out. We've created a survey, heavily inspired by the Decimal survey: https://forms.gle/iL9iZg7Y9LvH41Nv8

The idea is to get some basic idea of what people tried to do with Temporal and what parts they found easy or hard, and then also ask a few specific questions about the areas that we think may still need the most attention.

2

Hi @ptomato,

We will bee having a research call with Felienne on Thursday, it directly overlaps with your temporal meeting. That would be a good place to discuss your survey.

The only issue is the timing. Do you want us to adjust our meeting time so that you can attend?

3

Also, we already did a feedback round on the decimal survey and there were some changes. I will try to get those published.

4

I have done an initial review of the survey. I am not an expert, but here are my comments:

  • A couple of questions ask "How easy was it" --> these can all be changed to multiple choice. It will make it easier to analyze. This is true for a number of the long form answers. I made this mistake with my survey and it make the analysis part of the work really difficult. If it can be a prefab answer, it might make sense to use it. Also, sometimes surveys have an option to scramble the multiple choice, which can compensate for leading orders of answers.
  • For the question "How well do you feel you understand the Temporal API? Was it easy to understand?" -- it seems like the next question is very similar. I could imagine myself answering this, and then getting exhausted by the next question. Again, here a multiple choice could work but it depends on what you are driving towards.
  • "Did the Temporal types fit with the data model of your code?" maybe instead this could be "which types fit with the data model of your code", with a choice to select any number. It could be constrasted with "which didn't" and if they pick a couple, have an optional question that asks for more detail
  • I would recommend moving the name/company/confidentiality questions to the top.
  • you might want to add an email contact in the description. There is the repository, but the last time I ran a survey this was one of the comments I got back.
1 Like
5

Thanks very much for the feedback @yulia. I've updated the survey, especially since we didn't have any responses yet :smile:

You're right, I ended up just deleting this one.

great idea, I've done this! This will hopefully give us more effective data on, for example, the question of whether we really need Temporal.YearMonth and Temporal.MonthDay (which has been raised a few times over the history of the proposal.)

Can you tell more about what was difficult about it? Originally we thought that we wanted more free form answers so that we could get a qualitative understanding of how people feel about the API. (And to be honest I don't expect we'd get that many answers, since few people will invest the time in actually writing code with Temporal; but maybe my expectation is off here?)

6

Thanks for the analysis this morning, and sorry I had to leave before Records & Tuples. I personally found it really valuable and I've already addressed in the survey some of the easier points that were brought up.

I took notes until the point where I left and have now gone over them to turn my frantically typed txtspk into full sentences. Surely I missed a few things because the conversation went really quickly, but is there somewhere that people would prefer that I post them in order to combine with the notes taken during Records & Tuples?

I'd like to continue the discussion about what is the research question. I've given it a bit more thought since this morning. I think a the research question running through most of the survey is, "Given the current set of features of Temporal, are there any APIs which tend to confuse people in practice?" I think this is the main question that we want to answer before asking for a Stage 3 review. In the champions group we have found that we got the most valuable feedback on that topic from people who did spend a lot of time trying out Temporal.

Case in point being Justin who was invited to join after giving extensive feedback from a first-time-user perspective. At least for me personally, the goal was to repeat that experience, say, 10 or 15 times. This is also why we aimed for majority free-form questions, because so far our experience has been that just letting people talk / rant about Temporal has been valuable. However, I do want to be mindful of the pitfall (I think Felienne mentioned it; sorry, it was just moments before I left, so I didn't catch it in the notes) where we do get the kind of answers we want but they don't reflect reality.

A secondary research question is that we want to collect information on a few specific parts of the API, either ones that we aren't sure should be included or not (like the question about a type that combines a timestamp and time zone) or ones that we aren't sure we've made the right tradeoffs (like the question about withCalendar and toLocaleString). I'm not sure, should we split up the survey into separate surveys for each of these? On the one hand I take to heart Felienne's point that any question which is not related to the main research question detracts from the other questions, but on the other hand if we are only expecting a few people to have invested the time, wouldn't we be giving those few people survey fatigue?

7

Justin came up with a better wording of the research question: "Given the current feature set of Temporal, is there any common task that is hard to do?"