How can artificial intelligence help us to create visual content? This seminar course surveys recent research using AI and machine learning techniques to create 2D and 3D graphics content, with an emphasis on probabilistic generative models. Each week, we will read two papers on a closely-related topic and discuss their contributions, connections, and limitations. Students will also undertake an open-ended final research project. Throughout, the course will emphasize key academic skills such as critical paper-reading and how to give clear and compelling presentations.
For questions, discussion, and other course-related posts, use the Slack channel.
If you have an atypical question that you are certain does not belong on the Slack channel, email the instructor.
The instructor will not respond to direct messages on Slack.
Submit notes on the readings by 10am (and bring your notes to class).
Rough sample reading response that we wrote together in class (and my version).
- Make it Home: Automatic Optimization of Furniture Arrangement. SIGGRAPH 2011.
- Example-based Synthesis of 3D Object Arrangements. SIGGRAPH Asia 2012.
- Learning Design Patterns with Bayesian Grammar Induction. UIST 2012.
- Bayesian Grammar Learning for Inverse Procedural Modeling. CVPR 2013.
- Inducing Probabilistic Programs by Bayesian Program Merging. arXiv 2011.
- Interactive Design of Probability Density Functions for Shape Grammars. SIGGRAPH Asia 2015.
- Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. NIPS 2016.
- GRASS: Generative Recursive Autoencoders for Shape Structure. SIGGRAPH 2017.
- Unsupervised Creation of Parameterized Avatars. arXiv 2017.
- Crowd-driven Mid-scale Layout Design. SIGGRAPH 2016.
- Learning from Simulated and Unsupervised Images through Adversarial Training. CVPR 2017.
- Computer-generated Residential Building Layouts. SIGGRAPH Asia 2010.
- Material Memex: Automatic Material Suggestions for 3D Objects. SIGGRAPH Asia 2012.
- Magic Decorator: Automatic Material Suggestion for Indoor Digital Scenes. SIGGRAPH Asia 2015.
- Learning a Manifold of Fonts. SIGGRAPH 2014.
- A Neural Representation of Sketch Drawings. arXiv 2017.
Students who complete this course will:
- Acquire skills and experience for critically reading and discussing technical research papers.
- Develop techniques for giving engaging presentations of technical material.
- Be familiar with a broad range of state-of-the-art uses of AI for visual content creation.
- Build depth of knowledge in one technical area by conducting their own research projects.
- Gain experience in how to organize, execute, and write up a collaborative, open-ended research project.
Final grades in the class break down as follows:
- 20% Reading summaries & discussion questions
- 20% Presentation
- 20% Discussion
- 40% Final project
The subsequent sections of this page describe each category in detail.
The ability to read an academic paper and distill its main insights and contributions into a concise form is a critical skill for researchers. Reviewers for journals and conferences are often asked to do just this. For example, here is the first prompt on the SIGGRAPH reviewer form:
"Briefly describe the paper and its contribution to computer graphics and interactive techniques. Please give your assessment of the scope and magnitude of the paper's contribution."
Each week, you will submit two such summaries, one for each paper that we read. In your summaries, focus on concisely capturing the problem the paper is trying to solve, its key insights and methodologies, and its main results. Briefly assess the extent to which the paper meets its own goals and state its limitations.
In addition, you will also submit at least two thoughtful questions you would like to have answered about the readings for our in-class discussion.
Finally, you will submit an idea for a research project that is related to / inspired by the readings. This can be an extension of one of the works described in the readings, a way to address limitations, an application of the methods described to a new problem, a different approach to solve the same problem, or anything else that comes to mind. Don't worry about whether your idea is "good enough" or spend a lot of time trying to perfect it---the point is for the class to generate lots of ideas and to refine them as as a group.
Reading respones, discussion questions, and research ideas are due by 10am of each class meeting day. Submit them using this form.
Each class session, students will given in-class presentations of the weekly assigned readings. Depending on enrollment, this may either be two ~30 minute presentations (one per reading), or one slightly longer presentation. Presenters should aim to communicate to the audience (i.e. the class) the problem(s) addressed by the work, the state of previous work in that area, the main insights and techniques used, the most important results, and any limitations or weaknesses. Presenters should also take some time to highly connections between the two readings for the week (and/or to readings from prevous weeks): how are they similar/different? Does one build upon or extend another? What are their relative strengths and weaknesses? The use of images/diagrams/videos/code demos from the work being presented is strongly encouraged.
Following each presentation, there will be a short (~10 min) period for the class to give the presenter feedback on what worked well with their presentation, what didn’t, and how it might be improved.
Students are strongly encouraged to do a "practice talk" before their in-class presentation. This can be as simple as sitting down with a friend / roommate / classmate and giving your presentation to them. Running through a presentation with an actual, live audience member can highlight issues with pacing, clarity, etc. that go unnoticed when you practice on your own. The instructor is also available in office hours to help students with their presentations.
After each presentation, the class will discuss the presented work as a group. All students are expected to have read both assigned papers and to be prepared to discuss them. The purpose of the discussion is to dig deeper into the papers presented: to clear up any confusions people may have about methods, evaluate strengths and weaknesses, and (especially) to brainstorm potential extensions and further experiments that may serve as the basis for final projects.
Each week’s discussion will have an assigned student moderator. The moderator is responsible for keeping the discussion on-topic, to ensure everyone who wants to speak has an opportunity to do so, and to introduce new topics from the list of student-submitted discussion questions. The Sheridan Center for Teaching and Learning offers tips on how to lead an effective group discussion.
During the course, students will also undertake a final project to be presented at the end of the term. Projects are open-ended: they may further investigate methods or results from one or more of the papers discussed, develop an extension to or address a limitation of one of these papers, or pursue a new idea involving the use of AI for content creation. Projects will be done in groups of 1-4 students, with the scope of the project calibrated appropriately for the group size. For general inspiration, I recommend Tom Funkhouser's "Five Principles for Choosing Research Problems in Computer Graphics" (talk video, slides).
Final project timeline:
- Nov 3: Form project groups and propose projects. Project groups submit short proposals describing what they plan to do and how they plan to divide this work between group members. This step can be done sooner, if your group has a project in mind earlier in the term and wants more time to get it done.
- Nov 22: Status report. A short report describing the progress your group has made, what steps remain to be done, and any challenges or issues your group currently faces.
- Dec 6: Project presentation. Project groups present their results in class.
- Dec 15: Final report. A short document in the style of the papers we have read throughout the course. Use the SIGGRAPH ("acmtog" article type) format. Use of LaTeX is strongly encouraged: if you are not yet familiar with it, learning it is a good investment in a tool you will use throughout your research career. Reports should also include an additional page describing each group member’s specific contributions to the project. This need not align perfectly with the project proposal (as research projects naturally change and adapt in response to new challenges).
See these guidelines for more detailed descriptions of what should be included in each of these milestones.
| Activity | Hours |
| In class | 26 |
| Assigned readings | 55 |
| Summaries & questions | 5 |
| Presentation preparation | 10 |
| Project work | 60+ |
| Project proposal/write-ups/presentation | 20 |
| Office hours & out-of-class discussion | 4 |
| Total | 180+ |
Our intent is that this course provide a welcoming environment for all students who satisfy the prerequisites. All members of the CS community, including faculty and staff, are expected to treat one another in a professional manner. If you feel you have not been treated in a professional manner by any of the course staff, please contact either the instructor (Daniel) or the department chair (Prof. Cetintemel). If you have a diversity issue, please contact Laura Dobler. We will take all complaints about unprofessional behavior seriously. Prof. Krishnamurthi has good notes on this area.
Brown welcomes students form all around the country and the world, and their unique perspectives enrich our learning community. To empower students whose first language is not English, and array of support is available on campus, including language and culture workshops and individual appointments. For more information, contact the English Language Learning Specialists at ellwriting@brown.edu.
Academic dishonesty will not be tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Brown Academic and Student Conduct Codes. For project work, feel free to build on third-party software, datasets, or other resources, as long as you credit them in your report(s) and clearly state what work is solely your own. As a general policy (for this course and for the rest of your academic career): if you use any idea, text, code, or data from elsewhere, then cite it.
Brown University is committed to full inclusion of all students. Please inform me if you have a disability or other condition that might require accommodations or modification of any of these course procedures. You may email me, come to office hours, or speak with me after class, and your confidentiality is respected. We will do whatever we can to support accommodations recommended by SEAS. For more information contact Student and Employee Accessibility Services (SEAS) at 401-863-9588 or SEAS@brown.edu.
Being a student can be very stressful. If you feel you are under too much pressure or there are psychological issues that are keeping you from performing well at Brown, we encourage you to contact Brown’s Counseling and Psychological Services CAPS. They provide confidential counseling and can provide notes supporting extensions on assignments for health reasons.
We expect everyone to complete the course on time. However, we certainly understand that there may be factors beyond your control, such as health problems and family crises, that prevent you from finishing the course on time. If you feel you cannot complete the course on time, please discuss with the instructor the possibility of being given a grade of Incomplete for the course and setting a schedule for completing the course in the upcoming year.
Thanks to Tom Doeppner and Laura Dobler for the text on accommodation, mental health, and incomplete policy.