How I learned CoreGraphics

Left: lowres image, Middle: original highres, Right: AI-estimated highres

About me

I am Michael Verges, a first-year Computer Science major at Georgia Institute of Technology. My history with app development goes back about four years. For the past 3 summers, I applied to attend Apple's World Wide Development Conference (WWDC) as a student scholar. I have attended WWDC16, WWDC17, and WWDC18 as a scholar.

Why I wanted to learn more about CoreGraphics

One of the most common Frameworks iOS programmers interact with is CoreGraphics. UIKit (the standard User Interface framework for iOS) uses CoreGraphics and CoreAnimation. Despite the common interactions between UIKit and CoreGraphics, I never explored many underlying features.

How I approached learning CoreGraphics

I entered a 36-hour hackathon (HackGT) with a peer. He had an idea for a Super Resolution framework that uses AI to enhance images. I knew this would require lots of graphics processing on the iOS side.

Challenges I faced

Using a fully convolutional neural network, the AI could only read a fixed input size. This meant I needed to extract image data in indexed squares, pass them through the AI model, and re-stitch them in an image buffer. Learning to read pixel data and perform operations on image buffers was intimidating at first, but soon became natural.

Memory management was also very difficult when dealing with raw image data. I had used some example code, but the simulators would crash after hitting 4GB of memory. It was important for me to optimize speed and memory. I cut out any middle-man operations to directly perform image operations within an allocated pixel buffer.

Key takeaways

Hackathons are a great environment to learn a new technology. Also, develop your own understanding of the source code. While it is useful to learn from examples and documentation, it is important to rebuild it for yourself to learn the framework and optimize its uses.

Tips and advice

Watch your memory. Never allocate more than you need to.
Documentation teaches more than sample code.

Final thoughts and next steps

I love CoreGraphics and have spent the past few months developing signal-processing based projects. Currently I am using pixel data to algorithmically predict image interpolation rates (digital zoom).