By Jason Hartline and Aravindan Vijayaraghavan, Northwestern University.

Screenshot from the podium of our gather.town classroom.

Due to the pandemic, Northwestern computer science courses for the Fall of 2020 were taught remotely. We co-taught our undergraduate Theory of Computation course in a flipped format on gather.town. It was a fantastic experience. Class time was much more interactive than the Zoom classes we had both taught previously. Also, the students liked that they could interact with other students, and the change (versus yet another Zoom class). It went well and we plan to repeat the experience with our winter courses.

The rough details are as follows.

Flipped-class format

The course was taught in the flipped-class format. We had pre-recorded lectures that the students had to watch before each class. Class time, on the other hand, was spent discussing concepts and working through exercises in small groups.

Videos and Exercises

The model for pre-recorded video was three 15-minute videos, though in practice it was more often two 30-minute videos. Each video had accompanying exercises (implemented using the Canvas Quiz feature). It was recommended for students to interleave the exercise questions with the videos as some exercises were designed to reinforce concepts so as to make subsequent videos easier to understand. They were encouraged to work together on these exercises.

Class on Gather.town

The class time was entirely on Gather.town. The classroom was organized with cabaret seating layout. Students virtually sat at four-person tables (though the capacity was not a hard limit). We gave presentations from a podium at the front of the room and students could ask questions from microphones in the middle of the room. Both the tables and the podium and microphone were enabled by gather.town video chat.

A replica of our classroom is available for self-guided tours. It is recommended to bring a colleague. This classroom was provided to us for beta-testing by virtualchair.net and similar ones are now available from them or you can build your own on gather.town. (Full disclosure: Jason Hartline is a cofounder of virtualchair.net.)

The 80-minute class time was split into two parts. The first half of the class comprised a recap of concepts from the videos and related discussion, and the second half had the students working in groups on a homework-style problem.

Class Part I: Discussion

For the class discussion, we started with a slide of 5-6 discussion questions. This slide was screen-shared from a podium in the virtual classroom. As students joined the class they were encouraged to begin discussing these questions with other students at their tables. After ten minutes we led a discussion of the questions at the podium, encouraging students to chime in with answers from their discussion groups. Our virtual classroom had two microphones among the cabaret seats that students could use to address the class. We also encouraged students to bring up any questions they had.

This part of the class was quite interactive, and it would also give us a sense of how well the students understood the material, and enabled us to reemphasize material accordingly.

Class Part II: Problem Solving

The second half of class was reserved for student problem solving in groups (at their tables). During the problem-solving session we would join tables of students to answer questions, help talk them through issues, and ensure that they were making progress.

Other virtual interactions

The gather.town space had two additional rooms: a study hall and an office hours room. The study hall featured shared whiteboards and was a place where students could meet up for discussions and group work (homework problems were assigned to students in groups of two). The course staff conducted office hours in the office hours room.

Meeting on gather.town was very convenient and meetings of the course staff were also conducted in the virtual office.

Video content in watch parties.

We scheduled video content to be played in watch parties for students to view together the night before class. However, perhaps due to initial technical difficulties, this feature was not utilized by the students.

Difficulties

The following were the main difficulties we encountered. (Configuring the space was fairly easy with the virtualchair.net automation.)

1. It was slightly awkward that we could not leave our screenshare at the podium at the same time as we joined group discussions at the tables. This could be addressed by logging into gather.town twice and using one login for screensharing and the other for discussions with students.
2. We did not establish a video-on policy and we regret it. While we wanted to respect student privacy, students should be fully engaged in discussions and full engagement warrants videos being on. Moreover, we attempted to grade student participation, but it was difficult to know who is talking when many of the students had their video off.
3. Gather.town does not have a simple mechanism for keeping track of participation of students. Our process was manual and difficult.

Student Feedback

The following quotes from student the student course evaluations that pertain to the flipped format and remote technology. Students were generally quite positive.

• “The flipped format worked really well for this material; it was really valuable to be able to discuss practice exercises with our peers during class.”

• “Gather.town was a fantastic choice and made me really look forward to attending this class online. Getting to talk and solve problems with teammates really helped me consolidate ideas. (And it was also really nice to be able to socialize a bit.) The flipped classroom style worked really well.”

• “After the first few weeks, this was certainty my best class in terms of adapting to remote; gather.town discussions were really great (so great my table often stayed after class to continue them!)”

• “I thought that the gather.town format was an excellent decision. Being able to discuss course topics with other students in the class definitely helped me consolidate ideas. The recorded video lectures were high quality, and the professors both did a great job leading discussion in class.”

• “It was done on gather.town, which was a bit rocky the first few weeks but got better by the end; really enjoyed discussing with other people about the exercises (wish more time was spent on them actually).”

• “The practice exercises with our peers were really helpful. Most of the video lectures were clear enough, but being able to discuss points of confusion with classmates was a great way to clear up questions.”

Conclusions

Overall it was a fantastic experience that we are looking to refine in subsequent course offerings. Our virtual gather.town space was for our class only, but it would be natural to use the same space for multiple classes offered within the same department and doing so might encourage more student meetings on the platform. Our idea of watch parties for students to watch videos together needs further adjustments and testing.

As part of the HDR TRIPODS program, NSF has funded the Institute for Data, Econometrics, Algorithms, and Learning (IDEAL).  The institute is codirected by Prof. Hartline and Prof. Vijayaraghavan with key programs being organized also by Prof. Khuller and Prof. Makarychev.  It is a collaboration between Northwestern, Toyota Technology Institute, and University of Chicago bridges faculty in CS, Economics, Statistics, Electrical Engineering, and Operations Research.   See the news release by McCormick.

The EECS Department has announced multiple postdoctoral fellowships in Computer Science.  These fellowships come with a mix of teaching and research responsibilities and a ideal for candidates who wish to strengthen both their teaching and research experience before going on the academic job market.  Successful candidates will teach one course per term and conduct independent research, collaborating as is most effective, with current Northwestern faculty and students.

One of the priority areas for these positions is algorithms.  The teaching component of this position would be the undergraduate algorithms or discrete math courses and an advanced elective in the fellow’s research area.

The 2016 ACM Conference on Economics and Computation and the World Congress of the Game Theory Society (GAMES 2016) were colocated at Maastricht University in the medieval town of Maastricht, Netherlands in late July.  The Northwestern theory group had an especially strong showing with seven papers presented and a public lecture!  Brief summaries are given below.

Ph.D. student Manolis Pountourakis presented two papers.  At GAMES, he presented his paper Optimal Auctions vs. Anonymous Pricing (joint with with coauthors Saeed Alaei, Northwestern Prof. Jason Hartline, Rad Niazadeh, and Yang Yuan).  The paper proves that eBay’s Auctions and Buy-It-Now (posted pricings) are within at least an e = 2.718 factor of the revenue optimal auction (which may be asymmetric and very complicated). This is a worst-case bound that is quite difficult to prove; in practice the eBay’s sales mechanisms likely do much better.  This paper previously appeared in FOCS 2015 and with this video of the talk.  At EC, he presented Procrastination with Variable Present Bias (joint with coauthors Nick Gravin, Nicole Immorlica, and Brendan Lucier).  Individuals attempting to complete a task often procrastinate, making plans and then failing to follow through. One well-known model of such behavioral anomalies is present-bias discounting: individuals over-weight present costs by a bias factor. This paper introduces a variant of this model where the bias factor varies across time, identifies a connection between the planning problem with variable present bias and optimal pricing, and uses this connection to bound the cost of procrastination.

Nima Haghpanah, a former Ph.D. student and now faculty member in the Economics Department at Penn State, presented two papers.  At GAMES he presented Multi-dimensional Virtual Values and Second-degree Price Discrimination (joint with Northwestern Prof. Jason Hartline).  This paper studies multi-dimensional mechanism design.  It develops a method for solving for multi-dimensional virtual values, the pointwise optimization of which gives the optimal mechanism.  It uses these virtual values to characterize the family of distributions where second-degree price discrimination (e.g., with a high- and low-quality version of a product) is not profitable.  At EC he presented  Sequential Mechanisms with ex-post Participation Guarantees (joint with Profs. Itai Ashlagi of Stanford and Constantinos Daskalakis).  The paper identifies optimal and approximately optimal mechanisms in dynamic setting in which a seller and a buyer interact repeatedly.  In contrast to standard settings, in this paper the outcomes of a mechanism must be acceptable to the buyer even after all uncertainty is resolved.

At the second Workshop on Algorithmic Game Theory and Data Science (a part of the EC program), Ph.D. student Sam Taggart presented Non-revelation Mechanism Design (joint with Northwestern Prof. Jason Hartline).  This paper shows how to better design Bayes-Nash mechanisms, e.g., with winner-pays-bid or all-pay semantics, in a repeated environment by “linking decisions” across the repetitions.  The designed mechanisms are asymptotically optimal with the number of repetitions that are linked.

At EC, Prof. Jason Hartline presented A/B Testing of Auctions (joint with Shuchi Chawla and Denis Nekipelov).  This paper shows how the counter factual revenue of a mechanism B can be estimated from the equilibrium bids in a mechanism C which is the convex combination of mechanism A and B.  For example, an auctioneer with multiple units can determine whether selling one unit or two units gives higher revenue by running the auction that probabilistically sells one or two units.  The methods applies generally to position auctions to both winner-pays-bid and all-pay semantics.

Adjunct Northwestern Prof. Nicole Immorlica presented two papers.  At the Workshop on Economics of Cloud Computing she presented On-demand or Spot? Selling the Cloud to Risk-averse Customers (joint with former Ph.D. student Darrell Hoy and Nikhil Devanur). In Amazon EC2, cloud resources are sold through a combination of the on-demand market, in which customers buy resources at a fixed price, and the spot market, in which customers bid for an uncertain supply of excess resources. This paper showed that such a dual market system improves upon key objectives when customers have heterogeneous risk attitudes: the welfare and revenue of its unique equilibrium is larger than that of spot markets alone and the efficiency is larger than only on-demand markets.

A slide from “Market Design without Money”

At a GAMES affiliated evening soirée with the Brightlands Young Professionals network Prof. Immorlica gave a Roy-Lichtenstein-themed public lecture entitled “Market Design without Money”.  As a market-designer, a primary goal is to allocate resources to individuals in a way that maximizes the total value of the community for the allocation.  Selling the resources is a great way to achieve this: if someone is willing to pay a high price for something, then they must have a high value for it as well.  However, in many settings of interest, payments are infeasible.  This could be due to repugnance, as in the assignment of public schools to children or cadaver kidneys to patients.  Or perhaps the market technologies can not support financial transactions, e.g., users of an online review system like TripAdvisor may not have bank accounts linked to their user accounts.  This talk explored using alternative incentives including risk, waiting time, and social status, in place of money to help optimize markets in such scenarios.

Great work to the presenters and coauthors of these papers!

Sam Taggart won the 2016 Northwestern EECS Best TA award for his work TAing EECS 336: Design and Analysis of Algorithms this Fall. The class tripped in size over the last couple years and Sam taught all 150 students in six discussion sessions. Even with this overwhelming workload he received incredibly high scores on the teaching evaluations with rave comments. Great work Sam!!