How can artificial intelligence help us to create visual content? This seminar course surveys recent research using AI and machine learning techniques to create 2D and 3D graphics content, with an emphasis on probabilistic generative models. Each week, we will read two papers on a closely-related topic and discuss their contributions, connections, and limitations. Students will also undertake a collaborative, open-ended final research project. Throughout, the course will emphasize key academic skills such as critical paper-reading and how to give clear and compelling presentations.
For questions, discussion, and other course-related posts, use the Slack channel.
If you have an atypical question that you are certain does not belong on the Slack channel, email the instructor.
The instructor will not respond to direct messages on Slack.
- Material Memex: Automatic Material Suggestions for 3D Objects. SIGGRAPH Asia 2012.
- Magic Decorator: Automatic Material Suggestion for Indoor Digital Scenes. SIGGRAPH Asia 2015.
- Make it Home: Automatic Optimization of Furniture Arrangement. SIGGRAPH 2011.
- Example-based Synthesis of 3D Object Arrangements. SIGGRAPH Asia 2012.
- Probabilistic Reasoning for Assembly-Based 3D Modeling. SIGGRAPH 2011.
- ComplementMe: Weakly-Supervised Component Suggestions for 3D Modeling. SIGGRAPH Asia 2017.
- A Probabilistic Model for Component-based Shape Synthesis. SIGGRAPH 2012.
- Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling. NIPS 2016.
- GRASS: Generative Recursive Autoencoders for Shape Structure. SIGGRAPH 2017.
- Metropolis Procedural Modeling. TOG 2011.
- Interactive Sketching of Urban Procedural Models. SIGGRAPH 2016.
- Learning Design Patterns with Bayesian Grammar Induction. UIST 2012.
- Bayesian Grammar Learning for Inverse Procedural Modeling. CVPR 2013.
- Inducing Probabilistic Programs by Bayesian Program Merging. arXiv 2011.
- Interactive Design of Probability Density Functions for Shape Grammars. SIGGRAPH Asia 2015.
- Image-to-Image Translation with Conditional Adversarial Networks. CVPR 2017.
- Real-Time User-Guided Image Colorization with Learned Deep Priors. SIGGRAPH 2017.
- Colorful Image Colorization. ECCV 2016.
- Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. ICCV 2017.
- Unsupervised Cross-Domain Image Generation. ICLR 2017.
- Generating Sequences with Recurrent Neural Networks. arXiv 2013.
- A Neural Representation of Sketch Drawings. arXiv 2017.
Students who complete this course will:
- Acquire skills and experience for critically reading and discussing technical research papers.
- Develop techniques for giving engaging presentations of technical material.
- Be familiar with a broad range of state-of-the-art uses of AI for visual content creation.
- Build depth of knowledge in one technical area by conducting their own research projects.
- Gain experience in how to organize, execute, and write up a collaborative, open-ended research project.
Final grades in the class break down as follows:
- 20% Reading summaries & discussion questions
- 20% Presentation
- 20% Discussion
- 40% Final project
The subsequent sections of this page describe each category in detail.
The ability to read an academic paper and distill its main insights and contributions into a concise form is a critical skill for researchers. Reviewers for journals and conferences are often asked to do just this. For example, here is the first prompt on the SIGGRAPH reviewer form:
"Briefly describe the paper and its contribution to computer graphics and interactive techniques. Please give your assessment of the scope and magnitude of the paper's contribution."
Each week, you will submit two such summaries, one for each paper that we read. In your summaries, focus on concisely capturing the problem the paper is trying to solve, its key insights and methodologies, and its main results. Briefly assess the extent to which the paper meets its own goals and state its limitations. Reading responses are due by 10am of each class meeting day. Submit them using this form.
In addition, you will also submit at least two thoughtful questions about the readings for our in-class discussion. Discussion questions are also due by 10am of each class meeting day. Submit them using this form.
Each week, a student (or pair of students, depending on enrollment) will give an in-class presentation of the weekly assigned readings. The presentation will be ~45 minutes long: not enough time to exhaustively cover all contents of both papers, nor would that be an effective presentation strategy. Instead, the goal of the presentation is to ensure that the audience (i.e. the class) understands the problem(s) addressed by the work, the main insights and techniques used, and the most important results. Particular emphasis should be placed on the relationship between the two papers: how are they similar/different? Does one build upon or extend another? What are their relative strengths and weaknesses?
Following the presentation, there will be a short (~10 min) period for the class to give the presenter feedback on what worked well with their presentation, what didn’t, and how it might be improved.
Students are strongly encouraged to do a "practice talk" before their in-class presentation. This can be as simple as sitting down with a friend / roommate / classmate and giving your presentation to them. Running through a presentation with an actual, live audience member can highlight issues with pacing, clarity, etc. that go unnoticed when you practice on your own. The instructor is also available in office hours to help students with their presentations.
After the presentation, the remainder of class time will be dedicated to discussion. All students are expected to have read every assigned paper and to be prepared to discuss them. The purpose of the discussion is to dig deeper into the papers presented: to clear up any confusions people may have about methods, evaluate strengths and weaknesses, and (especially) to brainstorm potential extensions and further experiments that may serve as the basis for final projects.
Each week’s discussion will have an assigned student moderator. The moderator is responsible for keeping the discussion on-topic, to ensure everyone who wants to speak has an opportunity to do so, and to introduce new topics from the list of student-submitted discussion questions. The Sheridan Center for Teaching and Learning offers tips on how to lead an effective group discussion.
During the course, students will also undertake a final project to be presented at the end of the term. Projects are open-ended: they may further investigate methods or results from one or more of the papers discussed, develop an extension to or address a limitation of one of these papers, or pursue a new idea involving the use of AI for content creation. Projects will be done in groups of 2-4 students, with the scope of the project calibrated appropriately for the group size.
Final project timeline:
- Oct 27: Form project groups and propose projects. Project groups submit short proposals describing what they plan to do and how they plan to divide this work between group members. This step can be done sooner, if your group has a project in mind earlier in the term and wants more time to get it done.
- Nov 20: Status report. A short report describing the progress your group has made, what steps remain to be done, and any challenges or issues your group currently faces.
- Dec 6: Project presentation. Project groups present their results in class.
- Dec 13: Final report. A short document in the style of the papers we have read throughout the course. Use the SIGGRAPH ("acmtog" article type) format. Use of LaTeX is strongly encouraged: if you are not yet familiar with it, learning it is a good investment in a tool you will use throughout your research career. Reports should also include an additional page describing each group member’s specific contributions to the project. This need not align perfectly with the project proposal (as research projects naturally change and adapt in response to new challenges).
See these guidelines for more detailed descriptions of what should be included in each of these milestones.
|Summaries & questions||5|
|Office hours & out-of-class discussion||4|
Our intent is that this course provide a welcoming environment for all students who satisfy the prerequisites. All members of the CS community, including faculty and staff, are expected to treat one another in a professional manner. If you feel you have not been treated in a professional manner by any of the course staff, please contact either the instructor (Daniel) or the department chair (Prof. Cetintemel). If you have a diversity issue, please contact Laura Dobler. We will take all complaints about unprofessional behavior seriously. Prof. Krishnamurthi has good notes on this area.
Academic dishonesty will not be tolerated. This includes cheating, lying about course matters, plagiarism, or helping others commit a violation. Plagiarism includes reproducing the words of others without both the use of quotation marks and citation. Students are reminded of the obligations and expectations associated with the Brown Academic and Student Conduct Codes. For project work, feel free to build on third-party software, datasets, or other resources, as long as you credit them in your report(s) and clearly state what work is solely your own. As a general policy (for this course and for the rest of your academic career): if you use any idea, text, code, or data from elsewhere, then cite it.
Brown University is committed to full inclusion of all students. Please inform me if you have a disability or other condition that might require accommodations or modification of any of these course procedures. You may email me, come to office hours, or speak with me after class, and your confidentiality is respected. We will do whatever we can to support accommodations recommended by SEAS. For more information contact Student and Employee Accessibility Services (SEAS) at 401-863-9588 or SEAS@brown.edu.
Being a student can be very stressful. If you feel you are under too much pressure or there are psychological issues that are keeping you from performing well at Brown, we encourage you to contact Brown’s Counseling and Psychological Services CAPS. They provide confidential counseling and can provide notes supporting extensions on assignments for health reasons.
We expect everyone to complete the course on time. However, we certainly understand that there may be factors beyond your control, such as health problems and family crises, that prevent you from finishing the course on time. If you feel you cannot complete the course on time, please discuss with the instructor the possibility of being given a grade of Incomplete for the course and setting a schedule for completing the course in the upcoming year.
Thanks to Tom Doeppner and Laura Dobler for the text on accommodation, mental health, and incomplete policy.