Goals:
- Combine your interests (code and not-code)
- Build something meaningful for yourself
- Experiment with a specific technology that interests you
- Flex your skills
- Learn some new techniques & revisit topics we've discussed in class
Deliverables:
- The code itself, uploaded to Canvas or hosted on your own website
- One or more of the following:
- A live version of your code running on a website
- A .zip file with the code that can be run locally
- A video recap of the code in action
- A live performance??
- Documentation:
- Screenshots and/or video screen recordings that show off the output and program features
- A description of the piece (and title if applicable). At least 1 paragraph about what you've created, what you learned, tools, techniques, and anything else that makes your project interesting. Tell a story.
- Documentation should ideally go into your portfolio or exist on the web somewhere. You host it on your own website, the p5js editor, YouTube/Vimeo, or otherwise.
🎤 Case study: Me+You - voice recognition, volumetric shapes, and 3D simulation in a real ML-driven install
- (Beginner) Use a friendly, commercially-available ML tool
- (Intermediate) Find or download a tool/library/service that someone else has created, and use it
- (Advanced) Download a tool that someone else has made, and train your own data set, then use that
- (Expert) Write your own ML tool using existing ML frameworks like PyTorch or Tensorflow
* Just using a ML tool that is ready to use can be an expert-level journey
How to install an ML tool:
- Some ML tools are fully hosted online and you don't need to install anything
- Some will be pre-packaged into a nice UI in a desktop app. This is rare
- Some have online notebooks that are ready to use, usually in Google Colab. This creates a virtual environment with a good GPU to use. You should be able to copy a notebook into your own acount and start using it. This will also usually cost money if you want a very good GPU.
- Some must be installed on your own computer. Sometimes a very nice GPU is required (look for GPU RAM requirements). Often Linux and Windows are the target platforms
- Many existing apps are now adding AI features
ML is really good at dealing with images, so many of the popular machine-learning tools are either image generators or analyzers. However, some other popular ML tools deal with text, audio, and other data types to do amazing, magical things. Some places to start, depending on your interests:
- Computer vision (image analysis)
- Image creation
- ComfyUI
- Diffusers like Midjourney, Stable Diffusion, Dall-E, Firefly
- Image/Video manipulation
- Super-resolution tools
- Frame interpolation tools
- Style transfer, pix2pix and other image-based generators & effects
- Palette extraction
- Text (or code) generation
- LLMs
- Code assistants
- Audio manipulation
See Learning With AI
- Find almost anything on YouTube! The AI community is very active here
- Intro to Large Language Models
- https://microsoft.github.io/generative-ai-for-beginners/
- https://microsoft.github.io/ML-For-Beginners/
- https://github.com/xnought/vae-explainer
For a living, up-to-date list (this section ages fast - check here first): creative-coding-notes#machine-learning
Beginner-friendly
Computer vision
More advanced
- AWS ML tools
- Machine Learning for Artists
- Demucs (audio stem separation)
- Hotpot
🔍 Further (older/historical tools - still interesting, less likely to come up live)
- PoseNet / BodyPix - mostly superseded by Mediapipe now
- VQGAN + CLIP / StyleGAN3 - early-generation image synthesis, worth knowing the lineage
- Spleeter - see Demucs above for the modern equivalent
- Synopsis
- List: mlart.co
- Sougwen Chung
- Mario Klingemann
- Ellie Pritts
- Kyle McDonald -
[Discrete Figures](https://research.rhizomatiks.com/s/works/discrete_figures/en/) - Helena Sarin
- Memo Akten
- Refik Anadol
- Charlie Gerard
- Derrick Schultz
- Sofia Crespo
- WIRED: AI Is the Future - But Where Are the Women?
- Women in Machine Learning
- Women in Machine Learning & Data Science: Boulder
- Feminist.AI
- NPR: Lack of diversity in AI development causes serious real-life harm for people of color
- Healthcare
- Computer vision
- Self-driving cars
- Helpful robots (or are they war machines?)
- Natural Language Processing
- Enhancing old recordings
- DrumBot - a digital bandmate
- Bias
- Bias in
creative tools - Bias in
generative AI - Predictive policing algorithms are racist. They need to be dismantled.
- Healthcare bias
- Maybe it's not as good as we think
- Safety
- Privacy
- Privacy, again *
- Carbon footprint :-(
- Deepfakes
- Job security
Not necessarily specific to ML, but exacerbated by ML. Who is building the software that we use?
- Bias against left-handed people
- Color film was built for white people. Here's what it did to dark skin.
- What my color-blindness taught me about design
- Women Work Better in Warmer Rooms, Study Says
- Women suffer needless pain because almost everything is designed for men
- Women suffer needless pain because almost everything is designed for men
- Why Can't This Soap Dispenser Identify Dark Skin?
- The racism of technology - and why driverless cars could be the most dangerous example yet
- What Domino's digital accessibility lawsuit means for compliance
- Cost Cutting Algorithms Are Making Your Job Search a Living Hell
Read:
- Some of the articles above
- Investigate some ML tools
Build something with a machine-learning tool
- Or start work on your final project and present that next week
- Present your 3D/Computer Vision sketches