Chronicle
Conversational AI and Note taking.
Chronicle was developed for my Master's thesis at UC Berkeley. During the semester, I managed the entire product lifecycle, which ranged from identifying a societal issue to conducting primary and secondary research, and from designing to developing both a mobile and a WatchOS application.
You can read more about my academic research and references used here ↗
Role
Academic
2023 / (4 Months)
Mentors
Tools
Figma, Adobe CC
SwiftUI, Python, Adobe CC
Overview
Chronicle is your ultimate tool for seamless logging and retrieval on the go. Designed for versatility, each note supports multiple input formats—text, audio, images, and documents—ensuring your ideas and information are captured effortlessly. With an integrated personal assistant, retrieving your notes is fast and intuitive, making Chronicle the perfect companion for staying organized and efficient, anytime, anywhere.
Motivation
In today's digital age, cognitive overload from excessive information and quick content consumption has led to "digital dementia," reducing memory and attention spans. This growing issue impacts personal life, work, and mental health, sitting at the intersection of technology and cognitive science.
Many note-taking apps offer basic features but fail to effectively aid in memory and quick information retrieval. They are more about storing notes than helping users easily recall important details, a crucial need for managing today's information overload.
Designing for Retrieval
How can we give note taking apps the power they deserve ?
Note-taking apps, originally digitizing pen-and-paper methods, have seen limited evolution. Mainly offering text and image entry with stylus support, they lag behind the evolving demands and possibilities of digital note-taking.
Research
What is the need for another note taking app ?
Product Breakdown
How do people log their lives ?
There are 4 major mediums in which people capture moments in their lives :
Text
Capture precise details and thoughts quickly through text for structured reflection and note-taking.
Images
Instantly record emotions and moments with images, making journaling visually engaging and effortless.
Audio
Speak your thoughts freely; ideal for personal, expressive journaling on-the-go.
Documents
Easily organize and reference detailed information, blending professional and personal journaling.
How do we help people do these more effectively and efficiently ?
Contextualizing Images
Currently, most phones, by default, have the ability to :
Identify subjects in an image
Capture location of an image
Contextualizing Audio
Currently, most phones, by default, have the ability to :
Record and Playback Audio
Capture location of an image
Indexing text and PDFs is simple, but extracting insights from images and audio poses challenges. Images often hold rich information, while audio enables effortless, uninterrupted note-taking. Here's how I tackled these challenges to make them more efficient and insightful.
Contextualizing Images
Using Computer vision and newer vision models like GPT-4, we can make most of images:
" In the image, John is captured at Jacobs Hall, UC Berkeley, wearing a black hoodie with "BLACK SHEEP" text and handling a small, circular object, likely related to his thesis. He is using a glue gun, a detail evident from the photo. Uncharacteristically, he sports translucent-framed glasses. The background suggests he's engrossed in a robotics project, featuring stacked sections, cables, and electronics. Additionally, a Sony tripod with a GoPro logo, also on his table, indicates he might be preparing for or has just completed a shoot. "
-Chronicle
Contextualizing Audio
Using speaker diarization models we can make the transcript more useful :
Ananth : Hey John what are you working on ?
John : I'm just working on my thesis dude. I plan on assembling all the electronics and doing a dry run.
Ananth : Sounds sick dude !! By the way, do you know when milestone 3 submission is due?
John : Oh Yeaaa, Yoon mentioned that the submission is due in 3 days.
Ananth : Oh shoot. Okay thank you and good luck.
-Chronicle
How do we connect these different datasets ?
The core of Chronicle's tech stack is RAG on top of existing LLMs. By integrating LLMs like ChatGPT-4, Mistral AI, Claude, Llama etc with user notes, note taking experience can be enhanced for the users. Notes are stored and indexed locally, using langchain/LlamaIndex allowing these AI models to retrieve information quickly and intuitively for the user.
Read more ↗
Final Artifact
My Chronicles
Navigation
Navigation bar was designed with an intention to help users retrieve and input information as fast as possible
Quick Reminders
A reminder section that triggered notification manager and helped people keep track of their relevant recent tasks with the help of AI.
Tools for faster input
Custom toolbar and keyboard extension
Custom keyboard extension and toolbar
Taking into all considerations from user research. I designed and developed a custom keyboard extension to help people use their mode of logging without having to look around for it. At anytime during the logging process, people would have access to the custom tool bar that would allow them to add information in multiple formats.
real time Image from camera, image from gallery, voice and documents,
Notification Center
AI is as good as the data provided to it. While building chronicle, I realised that for a note taking app to be truly helpful, it needs to learn and understand it's user. Often time having the AI to assume things is what leads to frustration among humans. Having human inputs time to time, helped build richer data to be of more help in the future.
Summary View
Detailed summary for each view
Once logged. You can go back into each note, and quickly glance through audio transcripts, image descriptions, Generated reminders, location of log and much more.
Your AI Assistant
Jarvis is an AI assistant designed and developed to help retrieve any information from your notes. It can read through images, go through audio transcripts, add to your notes and delete from your notes. All with natural language like you are talking to your friend!
Log on the go
Chronicle on Apple watch
A watchOS app integrated with Siri helps log a lot faster than having to pull out the phone everytime.
Timely Notifications
Chronicle was designed to deliver timely notifications on tasks, updates, and helping you revisit and refresh your memory on previous notes.
MISCELLANIOUS
Chronicle was developed in a span of 4 months. During the time, I went about trying multiple iterations. Going through multiple MVPs and ideas to help create the most ideal note taking app. Here is a couple of iterations and MVPs I developed as part of my milestones.
First look and feel of jarvis. One of the biggest challenges was with making it feel instant with short wait times.
The goal with the first few MVPs was to test if people find value in an AI powered note taking app.
Gallery
