An Insider's Look at DrupalCon Session Selection

The call for sessions for DrupalCon Vienna just closed. Now all of us who submitted a session get to play the waiting game, eager to find out whether our session was accepted. Ever wonder what goes on during the session selection process? Here's some insight.

Amber Matz and I (Joe Shindelar) had the awesome opportunity to be part of the programming team for DrupalCon Baltimore. Both of us served as local track chairs. Amber worked on the Horizons track and I worked on the Being Human track. We thought it would be fun to share some of our experience as track chairs helping with session selection.

The official session selection process is well-documented. It’s designed to reduce bias and provide as much transparency as possible. Hopefully this post provides some additional insight into how sessions are chosen at DrupalCon.

Who chooses the sessions?

The DrupalCon programming team is responsible for soliciting session proposals, selecting sessions from those submissions, and laying out the schedule for the week of DrupalCon. The programming team is made up of Drupal Association staff and community volunteers. The exact composition of the team changes for each DrupalCon, but always includes a mix of people new to the process as well as those with previous experience.

Sessions are primarily selected on a per track basis, with input from the entire programming team. Each "track team" is composed of one local track chair and two global track chairs. (It was like this for DrupalCon Baltimore when I was involved, but I imagine this can vary.) These are the people who give your session the “thumbs-up” or add it to the "thanks, but no thanks" list.

Local track chair

For DrupalCon Baltimore, Amber was the Horizons local track chair and Joe was the Being Human local track chair.

The local track chair has these responsibilities:

• Write a description for their track along with specific topic suggestions
• Respond to inquiries about the track or requests for submission review from the community
• Actively participate in weekly meetings
• Determine the method and criteria your track will use to evaluate sessions
• Read and evaluate all session proposals
• Determine if session proposals would be a better fit in a different track and communicate accordingly with other track teams
• Select session proposals for inclusion in the final program, along with alternates
• Communicate with speakers and offer support to review slides or practice presentations

In many of the above tasks, especially session proposal selection, the local track chair works closely together with their global track chairs.

Global track chairs

There are one or two global track chairs for each track team. These are people who have previously acted as a track chair for DrupalCon. In many cases, the global track chairs were previously the local track chair for the track in question. This allows for things like transfer of knowledge, cohesion between events, and the ability to provide some insight into what did or did not work last time. For Amber and I, as local track chairs, the global track chairs were our first stop for questions.

Global track chairs have these responsibilities:

• Provide support and guidance to the local track chair and help them complete all their responsibilities.
• Support the transfer of knowledge and experience from previous events.

Each track team was responsible for figuring out a process and plan that worked for them with the local track chair taking the lead. Local track chairs (and optionally globals) met with Amanda, the Program Manager for the Drupal Association, once a week in the months leading up to DrupalCon. Besides directing the programming-related tasks for DrupalCon, Amanda coordinates all the volunteers on the track team. Our meetings generally involved locals giving an update about the current status of things. One such update might be: "My track has 25 submissions so far, I've been in contact with a few people to get clarification, and I've been working with my globals to finalize the draft of our track blog post that's due next Monday. One of the people who submitted a session asked me about how much time they should plan to leave for Q&A. What should I tell them?"

Meetings ended with locals having a list of action items to complete.

• Joe: For our team, I was always at the meetings, and the other two attended when they could, but not every time. I would generally immediately follow up with the rest of my team, let them know what was discussed, and what our next tasks were. For actionable tasks, I would usually try and take a first pass at accomplishing it and then work with the rest of the track team to make sure we all agreed. For example, I wrote the track description and then they helped by providing feedback.
• Amber: My experience was similar to Joe’s. I usually time-blocked the time right before and right after our weekly meeting to work on tasks and communicate with my globals. I would always ask my globals to review any copy I had written or responses to inquiries from the community.

Submit early

Here's our number one bit of advice: submit your session well before the deadline. Submitting early is about more than just garnering favor with an over-zealous track chair. It gives them an opportunity to ask clarifying questions, provide feedback, and give your submission the attention it deserves.

Towards the end of the session submission process, this becomes impossible to do. The volume of new submissions is simply too high. Check out the graph below. It shows the total number of sessions submitted over time for the DrupalCon Baltimore CFP (Call For Presentations). The CFP was open for 2 months, but 75% of the submissions came in in the last 5 days. About 38% were submitted in the last 24 hours.

• Joe: As a track chair it is really exciting when people submit a proposal to your track. And I read many of them as they came in. I was just thrilled that people were submitting things. I also started to follow up with people after reading their session descriptions if something wasn't clear, or if I needed more info. Since they submitted early, they also had time to revise. Every track chair on the program team had stories about reading early submissions and taking the time to reach out to get clarification and improve the submission.
• Joe: I wish I could give the same amount of feedback for every session, but the reality is this can only be done if you submit early. If you wait until just before the deadline I simply can't read all the submissions as quickly as they come in. By the time I do get around to reading it, it'll be too late for you to make any changes.
• Joe: A couple of people reached out to me specifically after submitting their proposal asking for feedback. I was happy to provide feedback in order to help someone refine their proposal and make sure that it fits with my vision for the track.
• Amber: As a track chair, I was trying to envision the final program and imagine how each presentation would fit into it. This sort of exercise of imagination becomes difficult with the overflow of submissions at the end.
• Amber: I really appreciated early submissions that were thorough. What I didn’t like were early submissions from veteran speakers who didn’t fill out at least 1 link to a recording of a previous talk. Just one is all I needed. Don’t make me hunt down your previous talks just because you’ve spoken many times before at DrupalCon. My advice for veteran speakers is: don’t assume the track chairs know who you are and what your reputation is.

Read the track description

Each track has put together both a description of the track and a blog post, explaining the vision for the track and the things that we are hoping to cover. The list reflects both our ideas about what we think will be valuable for the community and specifically DrupalCon's audience, and even topics of personal interest. You can bet we're going to keep that list in mind when we're picking sessions. You don't have to follow the list, but know that we're likely trying to make sure we cover things on the list.

• Joe: Every time I sat down to start reviewing sessions I would start by first rereading the description I had written for the track. This helped me to get into the right mindset and to remember what it is we're trying to accomplish. That being said, I was also trying to be aware of the fact that other people are going to have some great ideas that I didn’t think of when I wrote the track description. This was especially true when I saw that there were multiple sessions submitted about the same topic -- a good indicator that there is community interest. (Amber: I totally agree!)
• Joe: Sometimes I found it difficult to qualify if something was a good fit for my track or not. As someone who has submitted to DrupalCon in the past, I realize that sometimes you have an idea for a great talk that just doesn’t fit perfectly into any of the predetermined tracks. There was a lot of talk about this amongst the overall program team and my suspicion is that in the future this likely to change a bit. Possible ideas include allowing the selection of multiple possible tracks or even moving to more of a tagging system and less of a one-to-one relationship between sessions and tracks.
• Amber: I depended heavily on my global track chairs for input on the track description. The nature of the Horizons track is such that you want to make sure you’re not rehashing “the new hotness” from last year or the year before. I also tried, with a tiny bit of success, to solicit the community for ideas through a survey. I had better luck soliciting folks from our sister company Lullabot for ideas about current trends in tech.

Write a good description

After having read a few hundred submissions you start to get a sense of who put some thought and effort into their descriptions and who didn’t. Things like spelling and proper grammar aren’t necessarily going to make or break your chances of getting accepted. However, attention to detail in your description indicates you’re likely to put the same effort into preparing an awesome session.

• Joe: When reviewing sessions I pretty much immediately discarded anything that I could tell right away didn’t have much effort put into it. There were a few with only one or two sentences in the description. There was even one where someone had left the default “copy goes here” values in the form fields. I suspect they meant to come back and fill in more later but then never did. Either way, if I can tell you didn’t put much effort into your submission it’s going to be hard to convince me that you’re going to put in the time and effort required to prepare a quality session.
• Amber: In my mind, a thorough track description describes the topic and what problem it addresses, describes what the presentation will cover, and at what level, and describes learning outcomes—what the audience member can expect to learn by attending this session. An absence of any of these elements earned a lower ranking. No matter your level of English proficiency in writing, spelling, or grammar, have at least one person read it over and give you feedback. But please do them the courtesy of running your copy through a spelling and grammar checker tool first.

25-minute sessions

Prior to DrupalCon Baltimore, all sessions were always 60 minutes long. Or rather, 45-50 minutes with time for Q&A at the end. In Baltimore, the program team decided to introduce the option to have 25-minute sessions. And we allowed you to choose when you submitted your proposal whether you intended for it to be 25 minutes, 60 minutes, or that you were willing to do either.

• Joe: I found it tricky whenever someone would select the "either" option. It forced me to try and imagine what the talk might look like in a 25-minute version vs. a 60-minute version. And, I had to assume the description was written to cover the 60-minute version. What would they cut? Would they cut or just compress? A few people left extra notes indicating their intent. But most did not. I don't think it swayed my opinion of the session much one way or another, but I tended to assume that any session indicated as suitable for both was probably intended to be 60 minutes.
• Joe: Another thing I thought about a lot related to 25-minute sessions was the fact that when we scheduled them we put 2 sessions in the same room back-to-back. So the audience was likely to be mostly the same group of people for both. And thus it made a lot of sense to try and pair them based on a similar theme.
• Amber: After sessions were selected and I reached out to the speakers in my track, I did sense a bit of nervousness (maybe even annoyance?) from speakers chosen for 25-minute slots, especially when they had chosen the “either” option. It is quite challenging to reformat a talk to fit in a shorter time-slot, especially when it has been developed as a 50-minute session.

Note: We learned our lesson here. When submitting a session for the upcoming DrupalCon Vienna, you can choose 30 minutes or 60 minutes -- not both.

Reviewing sessions

All of the sessions were placed into a spreadsheet with a tab for each track. Each track team (local, and 2 globals) was responsible for picking X hours worth of content for their track plus alternatives in case someone backed out. Beyond that, the requirements were somewhat minimal.

• Joe: The programming team as a whole had at least 2 meetings where we talked about ways to approach ranking/rating sessions. I found that this was an area where it really helped to have the chance to work alongside other people who had done this before. You could really see how experience and knowledge gained over the years was carried along. For the Being Human track we chose to go through and individually rank each session from -2 to 2, tag them with info about how they mapped to our track description and the big ideas they included. For example, “Imposter syndrome” was something we knew we wanted to cover, and there were multiple sessions about it. Tagging each one allowed us to more easily identify duplication, etc. I also left a LOT of notes for myself. Much of the initial work of selection was assigning a rating and making sure I could figure out later why I assigned something that rating.
• Joe: I reviewed things in multiple passes. Sometimes I just went through and tagged based on topics covered. Another pass through I was just thinking about the speaker(s) and their experience speaking. In order to keep a clear head and to give each submission the quality review it deserved, I would limit myself to spending no more than an hour or two at a time reviewing sessions -- which basically meant I spent an hour every day for about 2 weeks just reading and ranking. I initially tried reading a bunch all at once but quickly found that they were becoming jumbled in my head and I was having a hard time distinguishing one "Imposter syndrome" session from another. And that didn’t seem fair. I was worried that sessions I reviewed most recently would just sort of win by virtue of the fact that I could remember them. This approach -- spending a shorter time each day -- ended up working really well for me.
• Joe: I really wanted to make sure that the Being Human track was for the community and not just for me. For every session I accepted or declined, I would ask myself the question, “How will I justify this decision to the community?”
• Amber: The Horizons track got lots of interesting submissions. For us, the main question was, how well does this session fit into the vision we had for our track, as described in our track description and topic suggestions list. I found it helpful to assign subtopics to each session so that we could see where there was more than one submission on a certain topic and so that we could compare submissions that were covering pretty much the same ground. We wanted to have complementary sessions or even two competing approaches to the same problem but avoid duplication. We also wanted to have as many “subtopics” in our track represented as possible. This is where it was nice to have the option to choose 25-minute sessions. We were able to cover a lot more interesting topics in our track that way.
• Amber: Ranking was difficult. I broke the rating up into multiple categories and then took the average. We also calculated a standard deviation so that we could see which submissions we needed to talk about. Where there was a debate, it really came down to, “How well does this session fit into the vision for our track?” and "Do we think this is a good speaker, based on what we know about them?" The bottom line was always, “Can we justify this decision to the community?” Most of the work of ranking we were able to accomplish asynchronously on the spreadsheet. For matters of debate, we took to Slack or Google Hangouts to discuss. It was hard to find time for all 3 of us to sync up, so we did as much as we could asynchronously.

Inclusivity

When you submit a session to DrupalCon there are a couple of optional questions related to inclusivity. As part of our effort to increase diversity amongst speakers, you're asked to self-identify as to whether or not you've presented at a prior DrupalCon and whether or not you identify with any of the Big 8 social identifiers. This information isn’t intended to be used as part of the session rating/ranking process per se, but we do use it to make sure we’ve got a diverse group of speakers.

• Joe: In our spreadsheet we just had a yes/no for "first-time speaker" and "identifies with one or more social identifier". It came up a bit when thinking about a speaker's prior experience. There was one case where we had two very similar sounding talks, one by a new speaker and one by someone who had previously talked about the same topic. Both were ranked equally and as a tie-breaker we opted to go for the new perspective, especially since a recorded version of the other talk was still available from a previous conference.
• Joe: When ranking sessions I intentionally didn’t look at this too much. I was trying to initially rate sessions based on the session and the speaker’s experience. After we had completed our selection for the Being Human track I took some time to go back through and generate some basic stats, such as percent of new speakers, percent non-male speakers, and percent identifying with one or more social identifiers. Our thinking was if we felt that any of those numbers needed to be improved that we could re-evaluate our choices to make sure we were representing the whole community. But, we were quite happy and didn’t change anything. The Being Human track had a really diverse pool of speakers to choose from, which made our job pretty easy in this respect.
• Amber: While the Horizons track did have around 20-25% session submitters self-identify with one of the Big 8 underrepresented communities, I was the only woman to submit to and speak in my track. (It is standard to recuse oneself from rating one's own talk or others where being objective would be difficult.) I thought this was a bummer. Although I did make an effort to reach out to women in the PHP and VR communities in particular, I was not successful in recruiting any other women to submit to Horizons. So, there's plenty of room for community growth, in that particular respect.
• Amber: I was pleased with the mix of new-to-DrupalCon and veteran DrupalCon speakers in the Horizons track. I have always been a bit weary of seeing tracks dominated by the same speakers and I was glad to be able to include a number of folks new to DrupalCon, but who were still excellent experienced speakers. It wasn’t a criteria for selection, but just a reflection of the quality of submissions from both new and experienced DrupalCon speakers.

One speaker, one session

You can submit as many sessions as you like. Submitting more than one might help increase your chances of being selected. But, with a few exceptions, we were committed to choosing one session per speaker across all tracks. We did allow a speaker to have a solo session and participate in a panel. In two cases we allowed a single speaker to have two solo sessions, but only after contacting them to chat about it.

Because speakers can submit multiple sessions to multiple different tracks this lead to some initial speaker duplication between tracks which we had to sort out.

• Joe: For the Being Human track we ended up replacing two of our initially selected sessions because of speaker duplication. In one case it was a relatively easy decision; we had another session on the same topic that we liked equally well and had already had a hard time choosing between the two. The other one was kind of a bummer. I was personally really excited about the session so it was hard to give it up. But, the other track team made a stronger case for why the speaker should present the session in their track. It was kind of surprising to me how I could get sort of emotionally attached to a particular session! The program team worked really well together and I think did a good job of creating a schedule that was good for the community and not just for me, the track chair.
• Amber: This was a bit of a challenge for us in Horizons. We had a number of excellent submissions, all on topics that we were interested in, but by the same speaker. In addition, the Horizons track topics have some overlap with PHP and Front-End, and so even when we thought we were out of the woods as far as overcommitting a speaker, we ended up making some compromises with other track teams in order to solve this problem of not allowing one speaker to speak too many times. I suppose it does increase your chances of being selected if you submit many times and are in general an awesome speaker and ridiculously intelligent, but it was a challenge to get that all sorted out. A good challenge, though, because I think it’s good not to have a track dominated by a single person.

In conclusion

In conclusion, being a local track chair took a lot of time and effort. Overall, it was a positive experience. It was really interesting to see this side of DrupalCon planning and to gain insight into the process of session selection.