By Jason Hartline and Aravindan Vijayaraghavan, Northwestern University.

Screenshot from the podium of our gather.town classroom.

Due to the pandemic, Northwestern computer science courses for the Fall of 2020 were taught remotely. We co-taught our undergraduate Theory of Computation course in a flipped format on gather.town. It was a fantastic experience. Class time was much more interactive than the Zoom classes we had both taught previously. Also, the students liked that they could interact with other students, and the change (versus yet another Zoom class). It went well and we plan to repeat the experience with our winter courses.

The rough details are as follows.

Flipped-class format

The course was taught in the flipped-class format. We had pre-recorded lectures that the students had to watch before each class. Class time, on the other hand, was spent discussing concepts and working through exercises in small groups.

Videos and Exercises

The model for pre-recorded video was three 15-minute videos, though in practice it was more often two 30-minute videos. Each video had accompanying exercises (implemented using the Canvas Quiz feature). It was recommended for students to interleave the exercise questions with the videos as some exercises were designed to reinforce concepts so as to make subsequent videos easier to understand. They were encouraged to work together on these exercises.

Class on Gather.town

The class time was entirely on Gather.town. The classroom was organized with cabaret seating layout. Students virtually sat at four-person tables (though the capacity was not a hard limit). We gave presentations from a podium at the front of the room and students could ask questions from microphones in the middle of the room. Both the tables and the podium and microphone were enabled by gather.town video chat.

A replica of our classroom is available for self-guided tours. It is recommended to bring a colleague. This classroom was provided to us for beta-testing by virtualchair.net and similar ones are now available from them or you can build your own on gather.town. (Full disclosure: Jason Hartline is a cofounder of virtualchair.net.)

The 80-minute class time was split into two parts. The first half of the class comprised a recap of concepts from the videos and related discussion, and the second half had the students working in groups on a homework-style problem.

Class Part I: Discussion

For the class discussion, we started with a slide of 5-6 discussion questions. This slide was screen-shared from a podium in the virtual classroom. As students joined the class they were encouraged to begin discussing these questions with other students at their tables. After ten minutes we led a discussion of the questions at the podium, encouraging students to chime in with answers from their discussion groups. Our virtual classroom had two microphones among the cabaret seats that students could use to address the class. We also encouraged students to bring up any questions they had.

This part of the class was quite interactive, and it would also give us a sense of how well the students understood the material, and enabled us to reemphasize material accordingly.

Class Part II: Problem Solving

The second half of class was reserved for student problem solving in groups (at their tables). During the problem-solving session we would join tables of students to answer questions, help talk them through issues, and ensure that they were making progress.

Other virtual interactions

The gather.town space had two additional rooms: a study hall and an office hours room. The study hall featured shared whiteboards and was a place where students could meet up for discussions and group work (homework problems were assigned to students in groups of two). The course staff conducted office hours in the office hours room.

Meeting on gather.town was very convenient and meetings of the course staff were also conducted in the virtual office.

Video content in watch parties.

We scheduled video content to be played in watch parties for students to view together the night before class. However, perhaps due to initial technical difficulties, this feature was not utilized by the students.

Difficulties

The following were the main difficulties we encountered. (Configuring the space was fairly easy with the virtualchair.net automation.)

1. It was slightly awkward that we could not leave our screenshare at the podium at the same time as we joined group discussions at the tables. This could be addressed by logging into gather.town twice and using one login for screensharing and the other for discussions with students.
2. We did not establish a video-on policy and we regret it. While we wanted to respect student privacy, students should be fully engaged in discussions and full engagement warrants videos being on. Moreover, we attempted to grade student participation, but it was difficult to know who is talking when many of the students had their video off.
3. Gather.town does not have a simple mechanism for keeping track of participation of students. Our process was manual and difficult.

Student Feedback

The following quotes from student the student course evaluations that pertain to the flipped format and remote technology. Students were generally quite positive.

• “The flipped format worked really well for this material; it was really valuable to be able to discuss practice exercises with our peers during class.”

• “Gather.town was a fantastic choice and made me really look forward to attending this class online. Getting to talk and solve problems with teammates really helped me consolidate ideas. (And it was also really nice to be able to socialize a bit.) The flipped classroom style worked really well.”

• “After the first few weeks, this was certainty my best class in terms of adapting to remote; gather.town discussions were really great (so great my table often stayed after class to continue them!)”

• “I thought that the gather.town format was an excellent decision. Being able to discuss course topics with other students in the class definitely helped me consolidate ideas. The recorded video lectures were high quality, and the professors both did a great job leading discussion in class.”

• “It was done on gather.town, which was a bit rocky the first few weeks but got better by the end; really enjoyed discussing with other people about the exercises (wish more time was spent on them actually).”

• “The practice exercises with our peers were really helpful. Most of the video lectures were clear enough, but being able to discuss points of confusion with classmates was a great way to clear up questions.”

Conclusions

Overall it was a fantastic experience that we are looking to refine in subsequent course offerings. Our virtual gather.town space was for our class only, but it would be natural to use the same space for multiple classes offered within the same department and doing so might encourage more student meetings on the platform. Our idea of watch parties for students to watch videos together needs further adjustments and testing.

The Northwestern CS Theory group had three papers at the 61st Annual IEEE Symposium on Foundations of Computer Science (FOCS 2020), which was recently held virtually.

Northwestern Economics PhD student Modibo Camara, CS PhD student Aleck Johnsen, and Prof. Jason Hartline co-authored a paper “Mechanisms for a No-Regret Agent: Beyond the Common Prior“.  The paper analyzes a broad class of Principal-Agent games in economics that normally depend on common knowledge of a precise distribution over an unknown, payoff-relevant input, using instead online learning in repeated game play.  A key idea of the paper which describes (possibly externally-informed) Agents behaviorally with only a no-regret property is to extend previous definitions for “regret” to consider counterfactual sequences of the repeated play. Link to the talk.

Northwestern CS PhD students Yingkai Li and Aleck Johnsen, and Prof. Jason Hartline co-authored a paper “Benchmark Design and Prior Independent Optimization“.  The paper introduces a formal approach to the study of benchmark design, as a parameter of worst case algorithm design to be optimized.  Incorporating first economic properties to justify and measure benchmarks, the main result of the paper shows that benchmark design is equivalent to algorithm design when inputs are drawn independently from a common-but-unknown distribution.  Another main result solves a longstanding open question in 2-agent revenue auction design, which further serves as application for the benchmark design result. Link to the talk.

Prof. Aravindan Vijayaraghavan co-authored a paper “Scheduling Precedence-Constrained Jobs on Related Machines with Communication Delay” with Biswaroop Maiti (Northeastern), Rajmohan Rajaraman (Northeastern), David Stalfa (Northeastern), and Zoya Svitkina (Google). The paper studies the problem of scheduling n precedence-constrained jobs on m uniformly-related machines in the challenging setting where we have to account for communication delays. Communication delay is the amount of time that must pass between the completion of a job on one machine and the start of any successor of that job on a different machine. The paper shows both algorithmic results and lower bounds. This includes the first polylogarithmic factor approximation algorithm for this problem, superconstant integrality gaps, and bounding the advantage of duplication in these settings. Link to the talk.

As part of the HDR TRIPODS program, NSF has funded the Institute for Data, Econometrics, Algorithms, and Learning (IDEAL).  The institute is codirected by Prof. Hartline and Prof. Vijayaraghavan with key programs being organized also by Prof. Khuller and Prof. Makarychev.  It is a collaboration between Northwestern, Toyota Technology Institute, and University of Chicago bridges faculty in CS, Economics, Statistics, Electrical Engineering, and Operations Research.   See the news release by McCormick.

The Northwestern CS Theory group had two papers at the 58th Annual IEEE Symposium on Foundations of Computer Science (FOCS 2017), which was recently held in Berkeley, CA.

TCS Postdoc Huck Bennett had a joint paper with Alexander Golovnev (Columbia and Yahoo Research) and Noah Stephens-Davidowitz (Princeton). The paper, “On The Quantitative Hardness of CVP,” initiates the study of the fine-grained complexity of lattice problems, a study which is important to the rapidly developing field of lattice-based cryptography. As its main result, the paper shows strong hardness of the Closest Vector Problem (CVP) with certain parameters assuming the Strong Exponential Time Hypothesis (SETH).

TCS Prof. Aravindan Vijayaraghavan had a joint paper with Oded Regev
(NYU). The paper, “Learning Mixtures of Well-Separated Gaussians,”
studies the classic problem of learning a mixture of k spherical
Gaussian distributions. The paper tries to characterize the
minimum amount of separation needed between the components to
estimate the parameters (means) of the Gaussians, and presents lower
bounds and upper bounds towards this end.

The Computer Science Division at Northwestern University welcomes new faculty member Dr. Konstantin (Kostya) Makarychev as an Associate Professor, beginning immediately. Dr. Makarychev’s position is one of the ten new faculty lines in CS which were announced in June 2016.

Dr. Makarychev is a theoretical computer scientist working on approximation algorithms, beyond worst-case analysis, applications of high-dimension geometry to computer science, and combinatorial optimization for designing efficient algorithms for computationally hard problems.

Dr. Makarychev joins Northwestern from Microsoft Research in Redmond, WA (2012-2016) and IBM Research Labs in Yorktown Heights, NY (2007-2012). Further details of his background can be found on his personal webpage.

Please click here for details, and the announcement on Northwestern homepage.

The EECS Department has announced multiple postdoctoral fellowships in Computer Science.  These fellowships come with a mix of teaching and research responsibilities and a ideal for candidates who wish to strengthen both their teaching and research experience before going on the academic job market.  Successful candidates will teach one course per term and conduct independent research, collaborating as is most effective, with current Northwestern faculty and students.

One of the priority areas for these positions is algorithms.  The teaching component of this position would be the undergraduate algorithms or discrete math courses and an advanced elective in the fellow’s research area.

TCS Prof. Anindya De had a joint paper with Michael Saks (Rutgers) and Sijian Tang (Rutgers) in 57th Annual IEEE Symposium on Foundations of Computer Science (FOCS 16). The paper “Noisy population recovery in polynomial time” addresses the problem of recovering an unknown distribution on binary strings under noise. This problem is related to well-studied problems in learning such as learning mixtures of spherical Gaussians and product distributions. A manuscript of the paper can be found here.

Theory PhD students Abhratanu Dutta and Yiding Feng along with their teammate Ruohong Zhang finished first at the ACM-ICPC Midcentral Regional Programming Contest this year. They finished in 1st place out of 156 teams representing 56 different schools in total and have advanced to the ACM-ICPC World Finals in Rapid City, South Dakota from May 20-25, 2017.

The ACM-ICPC (Association for Computing Machinery – International Collegiate Programming Contest) is a multi-tier, team-based, programming competition. Headquartered at Baylor University, Texas, it operates according to the rules and regulations formulated by the ACM. The contest participants come from over 2,000 universities that are spread across 80 countries and six continents.

Details can be found here.

The 2016 ACM Conference on Economics and Computation and the World Congress of the Game Theory Society (GAMES 2016) were colocated at Maastricht University in the medieval town of Maastricht, Netherlands in late July.  The Northwestern theory group had an especially strong showing with seven papers presented and a public lecture!  Brief summaries are given below.

Ph.D. student Manolis Pountourakis presented two papers.  At GAMES, he presented his paper Optimal Auctions vs. Anonymous Pricing (joint with with coauthors Saeed Alaei, Northwestern Prof. Jason Hartline, Rad Niazadeh, and Yang Yuan).  The paper proves that eBay’s Auctions and Buy-It-Now (posted pricings) are within at least an e = 2.718 factor of the revenue optimal auction (which may be asymmetric and very complicated). This is a worst-case bound that is quite difficult to prove; in practice the eBay’s sales mechanisms likely do much better.  This paper previously appeared in FOCS 2015 and with this video of the talk.  At EC, he presented Procrastination with Variable Present Bias (joint with coauthors Nick Gravin, Nicole Immorlica, and Brendan Lucier).  Individuals attempting to complete a task often procrastinate, making plans and then failing to follow through. One well-known model of such behavioral anomalies is present-bias discounting: individuals over-weight present costs by a bias factor. This paper introduces a variant of this model where the bias factor varies across time, identifies a connection between the planning problem with variable present bias and optimal pricing, and uses this connection to bound the cost of procrastination.

Nima Haghpanah, a former Ph.D. student and now faculty member in the Economics Department at Penn State, presented two papers.  At GAMES he presented Multi-dimensional Virtual Values and Second-degree Price Discrimination (joint with Northwestern Prof. Jason Hartline).  This paper studies multi-dimensional mechanism design.  It develops a method for solving for multi-dimensional virtual values, the pointwise optimization of which gives the optimal mechanism.  It uses these virtual values to characterize the family of distributions where second-degree price discrimination (e.g., with a high- and low-quality version of a product) is not profitable.  At EC he presented  Sequential Mechanisms with ex-post Participation Guarantees (joint with Profs. Itai Ashlagi of Stanford and Constantinos Daskalakis).  The paper identifies optimal and approximately optimal mechanisms in dynamic setting in which a seller and a buyer interact repeatedly.  In contrast to standard settings, in this paper the outcomes of a mechanism must be acceptable to the buyer even after all uncertainty is resolved.

At the second Workshop on Algorithmic Game Theory and Data Science (a part of the EC program), Ph.D. student Sam Taggart presented Non-revelation Mechanism Design (joint with Northwestern Prof. Jason Hartline).  This paper shows how to better design Bayes-Nash mechanisms, e.g., with winner-pays-bid or all-pay semantics, in a repeated environment by “linking decisions” across the repetitions.  The designed mechanisms are asymptotically optimal with the number of repetitions that are linked.

At EC, Prof. Jason Hartline presented A/B Testing of Auctions (joint with Shuchi Chawla and Denis Nekipelov).  This paper shows how the counter factual revenue of a mechanism B can be estimated from the equilibrium bids in a mechanism C which is the convex combination of mechanism A and B.  For example, an auctioneer with multiple units can determine whether selling one unit or two units gives higher revenue by running the auction that probabilistically sells one or two units.  The methods applies generally to position auctions to both winner-pays-bid and all-pay semantics.

Adjunct Northwestern Prof. Nicole Immorlica presented two papers.  At the Workshop on Economics of Cloud Computing she presented On-demand or Spot? Selling the Cloud to Risk-averse Customers (joint with former Ph.D. student Darrell Hoy and Nikhil Devanur). In Amazon EC2, cloud resources are sold through a combination of the on-demand market, in which customers buy resources at a fixed price, and the spot market, in which customers bid for an uncertain supply of excess resources. This paper showed that such a dual market system improves upon key objectives when customers have heterogeneous risk attitudes: the welfare and revenue of its unique equilibrium is larger than that of spot markets alone and the efficiency is larger than only on-demand markets.

A slide from “Market Design without Money”

At a GAMES affiliated evening soirée with the Brightlands Young Professionals network Prof. Immorlica gave a Roy-Lichtenstein-themed public lecture entitled “Market Design without Money”.  As a market-designer, a primary goal is to allocate resources to individuals in a way that maximizes the total value of the community for the allocation.  Selling the resources is a great way to achieve this: if someone is willing to pay a high price for something, then they must have a high value for it as well.  However, in many settings of interest, payments are infeasible.  This could be due to repugnance, as in the assignment of public schools to children or cadaver kidneys to patients.  Or perhaps the market technologies can not support financial transactions, e.g., users of an online review system like TripAdvisor may not have bank accounts linked to their user accounts.  This talk explored using alternative incentives including risk, waiting time, and social status, in place of money to help optimize markets in such scenarios.

Great work to the presenters and coauthors of these papers!