Adding Services to Grocery for Mac with Catalyst and AppKit

The two things I was most excited about while sitting in the audience for the WWDC 2019 keynote were the hotly anticipated encore for the Mac Pro tower computer, and the hotly debated introduction of the Catalyst framework for building Mac apps with UIKit. As a life long Mac user, and previous owner of earlier Mac towers, the introduction of the newest (and perhaps final) giant tower Mac was super exciting. But in some ways, the introduction of Catalyst was more meaningful to me personally as an engineer. Even though I've used a Mac for my entire life, and been programming since high school, I had never built an app for the Mac before.

So when I set out to build a version of my app Grocery for the Mac I knew from the beginning that I wanted it to be as Mac-like as possible. That meant menu bar menus full of keyboard actions, arrow key navigation for lists, right click context menus, expected features like open recent files, and adopting the Mac look and feel with translucent sidebars, a native Mac layout, and supporting system Dark mode and accent colors.

Honestly, getting Grocery to work the way I wanted it to with Catalyst was a lot easier than I thought it would be. Almost all of the work necessary was work I already had to do to build a great iPad app. Everything past that point, like supporting the menu bar, translucent sidebar, accent colors, dock actions, file open menus, etc. fell into place one piece at a time in a fairly orderly fashion.

But this isn't actually an article about Catalyst itself. Rather, this is an article about the final step in the process to Mac-ass the hell out of Grocery by publishing actions to the Mac services menu.

That step started when I re-read an excellent article by Steven Troughton-Smith called Beyond the Checkbox with Catalyst and AppKit. I had read the article over a year ago, and in hindsight it's incredible to me that Steven wrote this literally the first week that Catalyst was announced. Despite reading it a few days after he wrote it, I really needed to go through the process of getting my app ready for the Mac before I was able to really appreciate the additional power it contains. Here's what it let's you do.

What Steven describes in that article is how developers can create an AppKit plugin for their UIKit app running on the Mac. Normally, the UIKit part of a Catalyst app has no access to AppKit and thus can only interact with the parts of the Mac that Apple exposes through the Catalyst layer. But sitting behind all of that is AppKit itself, and the plugin system is the key to unlocking it.

The process of creating a plugin has a few hoops to jump through. The plugin is a bundle that is built and embedded in the target app at compile time. The bundle contains essentially whatever code you want it to. I called my bundle "Catalyst Bundle" and called its primary class "Catalyst App".

class CatalystApp : NSObject, CatalystDelegate {
    weak var callback : CatalystCallback?
}

The next step of the process is to actually load the bundle in your iOS app so that you can use it to talk to your plugin. Our bundle knows the name of its principal class, and also knows some information about its type (thanks to the Objective-C runtime). In my case, my principal class is a subclass of NSObject that conforms to an Objective-C protocol called "Catalyst Delegate". That way, I can create an instance of my principal class and cast it as a Catalyst Delegate. By storing a reference to that object in my app, I can treat it as a delegate to access functionality provided by the plugin.

class CatalystBridge {
    static func loadDelegate() -> CatalystDelegate? {
        #if targetEnvironment(macCatalyst)
        guard let url = Bundle.main.builtInPlugInsURL?.appendingPathComponent("Catalyst Bundle.bundle") else {
            return nil
        }
        guard let bundle = Bundle(path: url.path) else { return nil }
        guard bundle.load() else { return nil }
        guard let catalystBundle = bundle.principalClass as? NSObject.Type else { return nil }
        guard let delegate = catalystBundle.init() as? CatalystDelegate else { return nil }
        return delegate
        #else
        return nil
        #endif
    }
}

One other note about the bundle and protocol. I tried creating a Swift protocol of various forms (conforming to Class, NSObjectProtocol, etc.) but I was not able to get any of those protocols to work like this. Therefore, I resorted to declaring the protocol in Objective-C and including it in an Objective-C Bridging Header in both the plugin and my app target.

@protocol CatalystDelegate <NSObject>

- (void)setCallback:(nullable id<CatalystCallback>)callback;
- (void)setProxyIconFileURL:(nullable NSURL *)url;
- (void)setDockTileImageURL:(nullable NSURL *)url;

@end

Steven's article talks about all of the crazy things that an AppKit plugin like this lets you do. The first ones I tried involved overriding NSApplicationDelegate lifecycle methods. I wanted my app to quit after you closed the window: there's a delegate method for that. I experimented with changing the Dock tile, adding Dock menu actions, and adding a proxy icon in the title bar for the open Recipe file. I'm not shipping any of those changes yet because I didn't get everything to work the way I wanted to, but I am shipping the last thing that I tried which is publishing Services actions to the Services menu.

Services have been around on macOS X since the beginning, or close to it, but I only started using them personally fairly recently. Services are actions that operate on a type of content. The most common content types are text, images, files, and website URLs. I've created several services that operate on images and files as productivity tools. One of my favorites is a service to convert HEIC images into JPEGs (which should probably be built into the system at this point). I've got a few others for dealing with common types of files I use at work. When I saw Steven's mention of Services at the end of his article I knew I had to at least try it so I jumped in.

Trying to find documentation about how to define a Service action is virtually impossible. The most common search results all pertain to what we think of as "services" today, like iCloud or Apple Fitness+. The next most common search results are for user-facing ways to use Services, like Macworld and iMore articles for power users. The Apple tutorial for Services and this 2017 article by Kelvin Ng were the best documentation that I found but they still weren't enough to get everything working correctly for me. I'll try and fill in some of the missing pieces below.

A service definition starts with an entry in the Plist file. This is certainly where the bulk of Apple's documentation focuses, and most of it is pretty helpful. Under the NSServices key there's an array of service dictionaries. The most important keys in each service are the NSMenuItem, which defines the title of the service, the NSMessage, which defines signature of the method that will be called to activate the service, and the NSPortName, which is just the name of the app.

The other 3 keys that I made use of are the NSRequiredContext, NSSendTypes, and NSReturnTypes. The first lesson I learned is that NSRequiredContext needs to be defined with an empty dictionary before the service will appear at all - so that's step one. NSReturnTypes is only required if you plan on building a service that modifies data and sends it back, so if your service is only used for sending data from another app into yours then you can ignore that one.

<dict>
    <key>NSMenuItem</key>
    <dict>
        <key>default</key>
        <string>Add Items to Grocery List</string>
    </dict>
    <key>NSMessage</key>
    <string>addToList</string>
    <key>NSPortName</key>
    <string>Grocery</string>
    <key>NSRequiredContext</key>
    <dict/>
    <key>NSSendTypes</key>
    <array>
        <string>NSStringPboardType</string>
    </array>
</dict>

NSSendTypes was the trickiest for me to figure out. For text based services you only need to put NSStringPboardType in here. But I had trouble finding the right send types for a service I wanted to make that operated on files. I tried a few things that didn't work, but in the end I ended up checking the Info.plist file for BBEdit to see how they defined their "Open File in BBEdit" Service. They provide both the public.file-url and public.utf8-plain-text send types, which is what I ended up using for my Service (and so far is working great).

<dict>
    <key>NSMenuItem</key>
    <dict>
        <key>default</key>
        <string>Send Recipe to Grocery</string>
    </dict>
    <key>NSMessage</key>
    <string>importRecipe</string>
    <key>NSPortName</key>
    <string>Grocery</string>
    <key>NSSendTypes</key>
    <array>
        <string>public.utf8-plain-text</string>
        <string>public.file-url</string>
    </array>
    <key>NSRequiredContext</key>
    <dict>
        <key>NSTextContent</key>
        <array>
            <string>FilePath</string>
        </array>
    </dict>
</dict>

Back in my Catalyst App plugin class, I have a function that gets called after the app starts up that registers itself as the NSApplication's servicesProvider. By registering as the services provider I'm telling the system what class to send messages to when a service is activated.

extension CatalystApp {
    func setCallback(_ callback: CatalystCallback?) {
        self.callback = callback

        NSApplication.shared.servicesProvider = callback == nil ? nil : self

        NSUpdateDynamicServices()
    }
}

That also means that I need to implement methods on my Catalyst App class for each of the services that I defined in my Info.plist file. The method is named with the value I provided in the NSMessage key of my Info.plist file, and is passed 3 parameters: an NSPasteboard, a userData string, and a pointer to an error that can be displayed in the console if something goes wrong.

Since these have to be Objective-C compatible methods for the services system to send messages to them, I had a few hoops to jump through to implement them in Swift. Here's an example of what one of them looks like for the "addToList" Service that adds the selected text content to the grocery list.

@objc func addToList(_ pboard: NSPasteboard, userData: String, error: NSErrorPointer) {
        if let string = pboard.string(forType: .string) {
            callback?.didReceiveAddItems(toList: string)
        }
    }

Getting data out of the pasteboard and putting it back into the pasteboard is pretty much the same as it is in UIKit. The only two issues I ran into here were that the string representation of a fileURL needs to actually be converted into a real URL using the URL(string:) initializer before you can read it. Otherwise it's not useful.

@objc func importRecipe(_ pboard: NSPasteboard, userData: String, error: NSErrorPointer) {
        if let path = pboard.string(forType: .fileURL) {
            callback?.didReceiveAddRecipe(toGrocery: path)
        }
    }

The other issue was that adding items to the clipboard using setString: didn't actually do anything. I had to call clearContents() on the pasteboard first and then use the writeObjects method to add content back before it would actually replace my selected text in another app.

@objc func cleanUp(_ pboard: NSPasteboard, userData: String, error: NSErrorPointer) {
        if let string = pboard.string(forType: .string) {
            let returnText = callback?.didReceiveCleanUpRequest(with: string) ?? string

            pboard.clearContents()
            pboard.writeObjects([returnText as NSString])
        }
    }

The final piece of the puzzle for me was creating another delegate protocol for my plugin to communicate back to my iOS app to tell it when the Service received a new request. I created this as another simple Objective-C protocol to share with the same header file that I used earlier, and made my iOS app's app delegate conform to it. Now my plugin can send action requests and lifecycle events back up to my iOS app that it can pass through the appropriate methods that it already has access to. That works out nicely because all of the Services I created already map closely to NSUserActivity-based Shortcuts that my app supports, so the necessary code to handle them already exists.

@protocol CatalystCallback <NSObject>

- (void)didReceiveAddItemsToList:(nonnull NSString *)text;
- (void)didReceiveAddRecipeToGrocery:(nonnull NSString *)path;
- (void)didReceiveCreateRecipeWith:(nonnull NSString *)text;
- (nonnull NSString *)didReceiveCleanUpRequestWith:(nonnull NSString *)text;

@end

The end result for Grocery is 4 different Services that provide access to some of the most important features of the app:

  • Add Items to Grocery List - add selected text separated by newlines to the current grocery list
  • Clean Up Items using Grocery - modify the selected text using Grocery's CleanUp feature to prettify text
  • New Recipe with Selection - start a new recipe with the selected text
  • Send Recipe to Grocery - import a plain text file containing a Recipe into Grocery
Screen Shot 2021-02-24 at 7.58.18 PM.png

Unmodified Text

Screen Shot 2021-02-24 at 7.58.39 PM.png

Selected Service

Screen Shot 2021-02-24 at 7.58.46 PM.png

Cleaned Up Text

I'm really happy with how these Services turned out. I doubt this is a feature that many users would have asked for, but for me that wasn't the point of trying to build it. I just wanted to prove to myself that it was possible to build something supremely Mac-like in my Catalyst-based app, and I'm very happy to report that it is. And even more importantly to me, this brief little forray into AppKit has me even more excited about continuing to add more Mac-ness to Grocery in the future so that it will continue to be as Mac-assed as it possibly can be.

On Apple's SwiftUI Header File Documentation

I wanted to add my thoughts to Casey’s post about Apple’s developer documentation which itself stems from a really great episode of Under the Radar. I listened to the same episode and it helped crystallize the friction that I’ve been feeling with the process of learning to use SwiftUI in my work.

When I started learning to build iOS apps in 2010 I learned by reading tutorials and watching videos. At first the only things I knew how to do with a UITableView were the things I read about in tutorials. If I ran into a problem I didn’t know how to solve, I google searched until I found a tutorial that mentioned the name of the delegate method I needed to implement.

Eventually I learned enough of the patterns that iOS frameworks were built around that I realized I could probably just find the answer to my question in the built in documentation and header files. There were some great tools like Ingredients and Dash that helped me find what I wanted, but more often than not I relied on header files to solve my problem.

The beauty of that approach is that you’re always one command click / jump to definition away from discovering an answer. If you see the name of the type you need to learn about, or the signature of a method you have a question about, you can just click on it and start your research. You can see everything that type or method can do laid out in the header file.

I used that approach to learn the Compositional Layout system that Casey mentions. When I wanted to figure out how to make a sticky header in a scrolling list, I could command click through the definitions of a few types until the pinToVisibleBounds property of the NSCollectionLayoutBoundarySupplementaryItem class caught my eye. The answer was there for me to find after spending a few clicks to find it.

That brings us to today and learning to use SwiftUI. The Swift programming language may have gotten rid of the compiler’s need to define a real header file but it didn’t get rid of the programmer’s need to have clear and orderly descriptions of types and what they can do. When I started experimenting with SwiftUI my instinct was to start command clicking on types to learn from their definitions. This time though, command clicking didn't help me. When you jump to the definition of a type in SwiftUI you end up in the 22,000 line definition of SwiftUI itself.

Screen Shot 2020-11-10 at 10.38.06 PM.png

You actually can discover some of the modifiers and important types there that you need to use, but the lack of structural organization and the sheer scope of of the definition file keeps the information feeling like it's trying to hide from you.

I have yet to be able to solve any kind of non-trivial SwiftUI problem on my own inside of Xcode. For every issue I run into I’m falling back to the time-tested practice of google searching for a tutorial that happens to mention the name of a modifier that I need to know. The tutorials and new blogs and websites that I have found have been wonderfully helpful, but I don't believe they should be the only resources I can use to solve problems that I run into while using this framework.

I believe now that this aspect of SwiftUI is the most important issue standing in the way of me adopting it into a greater part of my workflow. I’ve now shipped quite a bit of SwiftUI code in the apps I work on, including in Widgets for iOS 14, but this friction in the process of learning SwiftUI takes away from the experience and doesn’t make me excited to learn more about it. It leaves me feeling like the incredible power of SwiftUI is trapped behind an invisible wall banging to be heard, but I just simply can’t see or hear it as I wander about concerned and trying to find it.

CloudKit Sharing for Grocery Households

Soon after Ryan and I released Grocery, our smart shopping list app for iOS and watchOS, we had a decision to make. We were already planning to add new features like support for recipes, so we had to decide what our app was going to be. Would Grocery always be a shopping list app, or would it become something else?

We decided then that Grocery would first and foremost be a shopping list app, and we don’t expect that to change. We’ve always kept that principle in mind while adding new features like recipes and later inventory tracking. We want anything we add to Grocery to make shopping better.

With that in mind, I’m really excited today to be launching a major new feature to Grocery that we call “Households”. We’ve been building towards Households for over a year now in order to support sharing almost all of Grocery’s features between family members.

Households enables shared sorting data, inventory status, and meal plans.

Households enables shared sorting data, inventory status, and meal plans.

Grocery actually launched 3 years ago with support for shared shopping lists through Reminders. Since Grocery keeps list content in Reminders, users can invite other people to a shared Reminders list and see all of the same content in their Grocery lists. That feature still works great and isn’t going anywhere, but we know that a lot of people want to share more than just the Grocery list content with their Family. That’s why we built our Households feature.

Creating a household allows families to share a common set of meal plans (planned recipes), stores (sorting order and shopping history), and inventory status (when things were bought and when they expire). All of this is made possible by Apple’s CloudKit Sharing feature which lets the household creator invite other people to join the household. That way, no one needs to create accounts or sign in with any kind of third party platform. Everything just works as long as everyone uses iCloud.

Grocery’s key feature is our smart sorting system that learns how to automatically sort your shopping list based on the order you shop in your stores. Sometimes people want to share the same sorted list order with other people in their family. If one person is creating the shopping list for another person to shop with, getting that list in the right order can help the person shopping be more efficient at the store. That’s a more important feature now than ever before, when people need to minimize the amount of time they spend in the grocery store.

When people are in a household together they’ll start seeing these kinds of behaviors:

  • When one person adds a recipe to the shared meal plan, everyone in the household will see that recipe

  • When two people are viewing the same store, the items will be sorted in the same order

  • Everyone in the household will get the same set of autocomplete suggestions

  • Everyone in the household will see the same purchase history and frequently purchased items

  • Everyone in the household will see the same in-stock items in the household’s inventory

The two behaviors that Households won’t cover at launch are Reminders list sharing and recipe folder sharing. Grocery still uses Reminders for the actual shopping list content, which means we still rely on the system Reminders app for actually sharing the Reminders list with other people. When people in the same family share a Reminders list from the system Reminders app, each person can see that shared Reminders list on their iOS device and in Grocery. 

We are planning to introduce our own version of iCloud shopping lists as an alternative to Reminders at some point in the future, but for now people that want to share list content still need to use the system Reminders app to invite people to shared lists.

We are also also planning to eventually support recipe folder sharing via the newly released iCloud Folder Sharing API, but since that feature just shipped we’re planning to wait a little while before adopting it fully.

We’ve found that sharing meal plans, shopping history, and inventory are helpful to how we use Grocery. When everyone in a household gets the same autocomplete suggestions, you get more consistently named list items. When everyone can see the meal plan, it’s easier to share recipes amongst people and to see what’s for dinner. When everyone can see the pantry contents it’s easier to answer the “should I add this to the list?” question before shopping.

Members of a household also gain access to all of Grocery’s premium features, so creating a household is a way for a family member to share Grocery Premium with everyone in their family without requiring everyone to buy their own premium subscription.

Household syncing is completely non-destructive. When multiple people join a household, everyone’s data is added to the shared household. If someone leaves the household they retain a copy of all of the data.

From a developer standpoint, building Households with CloudKit was actually a great experience. The mental model for sharing is all about parent references. When someone creates a household, all of their grocery data starts using that household as the parent record in CloudKit. That means all of their data needs to be uploaded back to CloudKit with the new parent reference, which can take some time. When you’re setting up Households for the first time it might seem like it’s going to take forever. I definitely wish it’s as faster to setup, but once it’s complete, incremental syncing is incredibly fast, with changes appearing on different devices usually within seconds. That’s a testament to how well the CloudKit subscriptions feature works which is what makes the real-time nature of this possible.

Since this release really is all about shopping we did also take time to fix several key bugs related to shopping and list sorting. We fixed two very hard to reproduce issues where the sorting history of some items could be forgotten, and an issue where adding new items to a manually sorted list could have strange consequences.

Last but not least, Grocery now supports PRINTED shopping lists! This feature has also been on our todo list since the beginning, and this felt like the right time to introduce support for it since having a printed, disposable, shopping list can actually help people stay safe while shopping at the grocery store.

Households are a big step forward for us and we’re excited about the potential. We’ll be continuing to improve how Grocery can be shared amongst families in future releases as well.

CocoaConf Yosemite

When I first heard about CocoaConf Yosemite I couldn't believe there was a conference designed around three very different things that I love: Apple, Photography, and the outdoors. I enjoyed the experience so much that I went back for seconds this year and had an even better time. The conference is returning for 2017, and I'm very much hoping to attend next year.

It's hard not to feel inspired in Yosemite National Park. Spending time outdoors has always been a great way for me to recharge, but the serenity you can experience in Yosemite Valley is quite unique. There really is nothing else like it. If this were just a tech conference then Yosemite would still be a very special setting that people would get a lot out of. But CocoaConf Yosemite is more than that. It's a place to focus on the human side of working in the technology industry and why what we do with technology matters to other people.

When I came to Yosemite last year I spent some time after the conference exploring the park. I had a chance to hike around by myself, which was a unique experience for me: I almost never go out hiking on my own. But it lead to some of the most amazing things I've had happen to me on the trail, like getting snowed in above Yosemite Valley!

Snow! I woke up around 6am with a foot of snow outside my little backpacking tent.

Snow! I woke up around 6am with a foot of snow outside my little backpacking tent.

Getting out and hiking by myself was actually pretty far outside my comfort zone. But that also fits the theme of the conference, which really focuses on learning more about yourself. Pushing myself lead some amazing experiences, including witnessing the most amazing sunset I've ever seen, above Half Dome. A picture of the sunset from Cloud's Rest is my favorite picture that I've ever taken. But it's not just the picture I like, it's the experience that it represents. Talking to people in the valley to get ideas on where to hike to. Planning the hike and determining if I could make it safely to where I needed to go. Being totally isolated, knowing I was the only one witnessing this in person up in the snow, but being able to share the experience with people later through photographs. And knowing the whole time that I was out there doing something new that I enjoyed. Such is the power of exploring the outdoors, that just one sunset can mean so much to you.

Sunset from Clouds Rest, overlooking Half Dome in Yosemite National Park.

Sunset from Clouds Rest, overlooking Half Dome in Yosemite National Park.

While I was exploring Yosemite that first year I had a few books with me. I read Becoming Steve Jobs by Brent Schlender, as well as Creativity Inc. by Ed Catmull. Both books taught me a great deal about leadership that I don't think I would have understood as well outside of a place like Yosemite. The conference and the environment were the perfect backdrop for listening to these types of stories.

This year leadership was a central theme at the conference, as well as another trait I wasn't expecting: vulnerability. I had started reading Daring Greatly by Brené Brown before the conference and I kept marveling at the similarities between what was being discussed in the conference talks and the themes of Brené's book.

Daring to be mediocre. The only thing between you and what you want to do is doing it. Be a little wild because we are in the wilderness. Be your own replacement. Challenge yourself. Having empathy for the point of view of others.

Many speakers told stories of using 31 day challenges as a way to coach yourself into doing something you were interested in but not comfortable getting started with. It takes courage to start something new, especially something you don't know you'll be good at. Daring to be mediocre helps you take a step towards something you want to achieve but are having trouble getting started with.

But the biggest takeaway I had from Yosemite this year was community, and how important it is. Sometimes it's easy to forget in the interregnum between WWDC's just how special the community is around the technology industry. The tech community is one that I think can be of immense service to others. I was so impressed by the talk Christa Mrgan gave on Civil Comments, a tool that is actually helping change people's behavior and positively influence the nature of discussion online. That's the type of impact we can have on the world, and I think it's a perfect example of why CocoaConf Yosemite is so great. It's about inspiring you to make a difference, in your team, in your company, in your community, in the world.

CocoaConf Yosemite is the best experience I've had at a conference. If you haven't been I highly encourage you to go. I'm hoping to return again next year.