Taking XML in computer science is like taking shop class in high school: You walk into it thinking it’ll be an easy “A,” but then quickly realize that it’s actual work when you pull the trunk that acts as a switch on your elephant lamp and the light doesn’t come on.
It would appear that Xcode in Mavericks now shows certain Scripting Bridge warnings in the console by default. I upgraded to Mavericks Friday, started a debug run of a SB-using project and this appeared out of nowhere:
2013-10-28 09:14:42.849 Ratchet[2387:303] warning: failed to get scripting definition from /Applications/Adobe Photoshop CC/Adobe Photoshop CC.app; it may not be scriptable.
2013-10-28 09:15:08.048 Ratchet[2387:303] warning: failed to get scripting definition from /Applications/Adobe InDesign CC/Adobe InDesign CC.app; it may not be scriptable.
This appears to be neither actionable or able to be disabled, so not much use during runtime. Not that it matters to me anymore anyway.
John Gruber linked to an article that explains the dropping of Applescript from the latest release of iWork.
The writing has been on the wall about this for a long, long time. How long has it been since Apple allowed their Applescript pages be allowed to go offsite? Instead is simply lamenting about the impending, far-too-long-drawn-out demise of Applescript, I wanted to substantiate a claim made by the aforementioned article:
What I suspect Apple doesn’t realize is how much small business and small shops workflow depends upon Applescript. Casual use is fine. But a lot of people do more.
iWork 13 — A Huge Regression
iWork is irrelavent in my industry; the lingua franca for developing book content is Microsoft Office, but the red flag being raised is just as relevant. Also, I am not a small company. I work for a large company that is almost a third Macs. But, I think there is the point to be made that Apple might not be aware of the complexity of workflows built around Applescript, and how casually and quietly deprecating support sucker punches any developer regardless of size.
At my day job in a publishing company, I manage a pre- and post-press department that is concerned with the archiving, retrieval, and most importantly re-use of content. (I suppose this could be summarized as “content lifecycle management.”) Content comes in many forms but for me the focus is on art and text.
For art, we have a number of digital products that are derived from the main texts that we publish, all of them an image bank in one form or another. An image bank is essentially a catalog of images that is easily searchable and useable by instructors and students ultimately showing an image on its own page. The term “image” in this context is actually comprised of several elements:
- The picture itself
- Callout (Figure 1-1, Figure 1-2, etc.)
- Relavent metadata, like page number
Image banks come in three forms, two web-based versions—one of which you can see an example of here—and PowerPoint. The PowerPoint-based image bank is literally each slide showing the above components for a given image.
An image bank can have anywhere between 200 to 3,000 images, and not all images have all of those elements. A book will have any or all the different types of image banks, so content must be displayed in each of those consistently. My team receives hundreds of varying types of content requests each month, but the image banks are the biggest project in terms of scale and scope by a huge margin. We do fewer image banks than we do of other requests, but image banks take the longest and are the most time consuming.
A long time ago, we used to make image banks by hand. The process took months and was riddled with errors since someone manually proofed and placed at least 1,000 content and data points. Then, a former employee wrote an Applescript and everything changed.
Applescript has helped us make the process much more reliable and faster. Essentially, the way the creation process works is the following:
- My team collects all content into an Excel spreadsheet templates (this is still done manually, though XML is going to change that really soon)
- Editorial edits the content to meet common and lowest-common-denominator image bank specs
- My team takes the edited spreadsheets and runs a script to create the product.
- The script extract the content from Excel, massage as required, and then transform into the required format
- The resulting files are returned for proofing and any changes made are done in the spreadsheet (they are almost always typos) and the process starts at the beginning again
The “resulting files” can be made in any number of ways, but Adobe Photoshop is a common tool because all art must be converted to meet web specs or be usable in PowerPoint. Adobe InDesign is another tool in the process because one of the image banks uses some fairly complex layouts.
The Applescripts do a lot for us in the process:
- Captions and credits are made “web ready” by converting special characters to HTML entities and removing any odd formatting that may have been carried over from the page layouts.
- Common text styling is captured from Excel and applied in the output files via a simple spec.
- Layouts are created and optimized dynamically via simple rules with no manual intervention.
- Files are appropriately named, errors logged, and so on.
Applescript makes inter-application automation a breeze. Most scripts go between the Finder, Excel, and then InDesign or Powerpoint at the click on icon, and I have developed scripts and processes that I can hand off to people with no programming knowledge of any kind. These scripts are anywhere between 1,200-3,000 lines of code; they are major applications unto themselves. My team doesn’t even open the script in Script Editor or Script Debugger to run it. I always felt the real power of Applescript was the ability to create complex workflows so easily, but that’s provided application developers are willing to support Applescript. In this case, Applescript saved the day and continued to for a long time.
Enter JSX, Wither Applescript
The first nail in the coffin was when Photoshop could no longer open a file via Applescript. But with the release of Creative Cloud and the associated updates to Creative Suite 6, now their dictionaries are stripped of the needed command to create a hybrid Applescript-JSX workflow. The final nail that came with Creative Cloud is that I have since discovered that the JSX framework is no longer accepting arguments as part of script calls from external commands. Everything just passes through as
undefined. This may be a big bug, but given the (very) niche nature of my environment, I don’t hold any hope this will be fixed.
I have been porting most of my Adobe automation code to JSX—rebuilding much of what I have done in Applescript in JSX—an effort that has taken better part of this year to get ahead of these changes. I had originally banked on Scripting Bridge to help with pushing content from Excel to the Creative Suite. My goal was to consolidate all of my significant workflows into a Cocoa application for ease of use and better long-term project management. But since that relies on the same frameworks as Applescript—see Apple’s Open Scripting Architecture that Adobe is clearly looking to drop—I now have to look into alternatives.
Again, from the article:
It wouldn’t be so bad were there an alternative.
iWork 13 — A Huge Regression
Alternatives to Applescript depends entirely on one’s own technology stack and a willingness to accept change. To put it another way, alternatives to Applescript simply do not exist only when one decides to stick with Mac OS.
I am exploring options in still maintaining consistent, reliable workflows to create image banks with Adobe Creative Cloud on the Mac. Right now, I am focused on the idea that since I am working with as much text as I am art, I can leverage tab-delimited files (a perennial favorite around here) saved from Excel and JSX’s baked-in XML parsing to create an content transfer spec by creating an in-house content transfer file specification. This means that one-button-click processes turns into n-button-clicks processes, but I still get to keep my investments in Mac OS. I’m not thrilled with this and neither will my team.
Another option is to remove Creative Cloud from the process completely, especially since we are moving everything to web anyway and layouts can be managed with CSS. The PowerPoints are a sticky wicket since that is one of the most-requested formats, and we are beholden to our customers. If not, I would jump to PDF in a second, making the slides using LaTeX. Convincing Sales and Editorial to migrate to PDF, however, would be a huge leap, but one that I am also considering for a variety of reasons, not just this.
Luckily for me, Adobe’s applications are cross-platform as is their JSX platform, and Adobe still maintains a Visual Basic API. The implication is that when Microsoft decides to drop Applescript support from Office as well—given that Adobe and Apple are moving in this direction, why should Microsoft continue?—I can make the move to Windows for process automation and use Visual Basic to automate Excel and then, with some relatively minor changes, carry over my JSX code as writ.
There are some serious caveats to significant of a migration, training and monetary investment notwithstanding. Primarily, if I can reliably call a JSX script with arguments via Visual Basic, then I have effectively re-established where I was before Creative Cloud was released. If Adobe has not treated Visual Basic with the same disdain as it has Applescript, then I could eventually be back where I was before JSX was released. That will take time, but in talking with our preferred vendors, the fact that we are using Macs for such large-scale production makes us the odd one out anyway. I will always have a MacBook Pro by my side for everything else, but then I lose my ability to effectively replicate my environment across both computers.
Switching platforms is something that I don’t want to do but the enormity of my Applescript code cannot be overlooked. At the same time, I have made significant investments in Mac-only applications and technologies—BBEdit, OmniGraffle, Cocoa framework—and now Unix now that I have been in school these past couple of years. There is still a lot of Applescript code running around here (and elsewhere) that is useful and viable.
But, when I do take a step back and survey the current environment as it pertains to publishing automation, there is little being done these days that is specific to Mac OS, and with the limited resources available to me I need to consider where I will receive the most support from the platforms upon which I am supporting my career. I’ll try a couple things first and then weigh my options.
Pretty much all of my blog posts about Adobe are critical, if not outright negative. This is because I care about the environment in which I work. My line of business, I am all but compelled to work with Adobe products. They are a major focal point of my development efforts, and are something with which I have had a relationship for over twenty years now.
Publishing technology has few alternatives. Before InDesign, QuarkXpress was the dominant application for page layouts. When InDesign CS2 was released, a gust of fresh air blew into the state of page layouts, and Quark quickly was given a run for its money, quickly outpaced by InDesign’s innovations. There were others, like PageMaker and FrameMaker, but they had their specific markets and requirements that generally didn’t meet book publishing’s general needs with the same ease as Quark and InDesign.
For a case in point, one need only look at the link manager in both applications. A “link” in this case is a piece of art placed into pages, or an XML file used to populate a template. A lot of time has passed since I worked with Quark in depth, but I remember Quark’s link manager being very linear with a poor UI and file selection UX. InDesign’s link management became everything that Quark’s wasn’t and kept innovating. The link manager is not “sexy” in the sense that it is an easy-to-implement, eye-popping effect for a client to see, but it is a focal point in the application for the page builder, who has to manage anywhere between a handful to thousands of links in a single job. A designer cannot build a design without using the link manager; it is like needing food to survive. With Adobe Creative Suite 4, my company made the decision to have all new titles be made in InDesign and have yet to change that policy. I learned how to script on that version and have built up a substantial library of code since.
Recent releases of the Creative Suite, and the newly released Creative Cloud, have left me wanting for better Applescript support, to the point where much of my code is going to be rendered obsolete within a year at the rate things are going. In my team’s recent purchase of Quark licenses—purely for legacy file support purposes, sadly,as we have a soft spot for Quark since some of us built are careers on it—the link manager was the first feature we looked at. It was immediately apparent that their link management had not changed one iota since InDesign’s first release, and quickly went back to work with InDesign, disappointed that nothing new was on the horizon for us.
There is one promising open source page layout application, but since scripting support appears to be even more dodgy than InDesign’s, there is no point in even downloading the application. “Contribute to the project,” you suggest? Noble, but there is no way I can fit in free development work in between The Day Job, school (I get 10-20 hours of homework each week), and family. So, I keep slogging with InDesign.
Really, the only way I can get out from Adobe’s mire would be to change my careers away from anything remotely design-related, which wouldn’t be all that bad, but that would really mean changing industries, which is something that is, personally, unappealing to me for the time being. I have my reasons, but this is not the forum in which to discuss them.
What this means is that, in the end due to industry and personal constraints, I cannot avoid Adobe, and will not be able to in the foreseeable future. While I am here, working with their apps, I have a desire to see them improve, or at least be better maintained. Adding in 3D object creation into Photoshop is not what I consider a good use of anyone’s time. So, I criticize.
I have this working draft of a post explaining how badly broken Applescript is from purely an Apple-owned technology perspective, and imploring Apple to do something about it. It’s a bit long. But, since Adobe’s Creative Cloud has been released and I got my grubby little hands on a copy, I decided to look at the state of Applescript support in my most-used applications. It’s not that I have high hopes of anything improving given support since Creative Suite 5, but this is my bread and butter so it’s worth a look to see what I have to workaround in future upgrades of scripts, if upgrading a script is even worth the time.
Currently, my Applescript “support” of in-house that utilize Adobe applications has been relegated to using one call—
Here is what we have to work with.
- Photoshop: Line 1280.
- InDesign: Does not exist in the header.
- Illustrator: No header could be generated because the sdef (scripting dictionary) file does not exist.
I took a look at some other apps, like Bridge, and things go generally downhill from there. Essentially, any Applescript-related support is rendered useless in Creative Cloud. This is infuriating, back-stabbing bullshit. I would also like to say this is also unacceptable but that would ring hollow since the state of publishing technology is such that I have no choice but to use Adobe software. Nowhere on their site do they formally announce the drop in support in Illustrator, and given the dodgy performance of their Applescript API.
I wish Adobe would do three things:
- Formally announce that Applescript, if not OSA support in general, is going away.
- Pick a date well in advance (like a year) for developers to prepare.
- Turn it off all at once so I can stop having to hunt for the state of their API.
This unannounced erosion in API support by inconsistency is poor form, though I expect nothing less from Adobe these days, and reflects their general customer support as well.
The actual voting is likely not very scientific, but this is a really great collection. I’m a bit surprised to see that (a) Source Code Pro is at the top (by a thin margin) and (b) there is no italic form of the font. That latter point is a glaring and non-starter omission for me as I change all my comments to italics to help with reading context, and doubly so since this an Adobe font, a world leader in font development.
I had forgotten about Anonymous Pro, which I liked for a while, but I thought the glyphs were too wide.
I am currently using Menlo and really like it, particularly at 11 pt. Most of my editors default to Monaco, which I never liked. Monaco always just seemed a bit too clunky for my taste.
Subject says (most of) it all. I have been working with the command line more lately, partially out of need, largely out of preference, and am starting to collect a bunch of little snippets I have modified from other or created out of whole cloth to get me through my day.
The section has a germinal collection of command line snippets, but the real work lies in the shell scripting cheat sheet and info on creating a command line application with Xcode using Foundation classes.
Friends don’t let friends drink and drive.
Friends don’t let friends drink and drill.1
Friends don’t let friends drink and Photoshop.
Friends don’t let friends drink and review code.
1 Apologies to Matt Groening, but this has always been one of my favorites.
Working with Excel files in Excel via Scripting Bridge is non-obvious, even when the bulk of the work done is extracting information from Excel like I do. Exporting content from Excel into CSV or a tab-delimited file isn’t an option since oftentimes I need to capture styles applied to text that is simply lost upon export. I have to work in Excel, and over time I have learned that in order to work with an Excel file with Scripting Bridge, the file must meet these conditions:
- Saved in
.XSLXformat and not
- Have more than one worksheet (tab).
- Not have any data connections.
The first condition was simple to troubleshoot and check because the file will not open. There is no visible error, but the returned value is
nil. Bam. Done.
The second condition was not as obvious. I discovered the condition through the
getItemCountInColumn:removeHeaderRow:inSheet: function in my
OCScriptingBridgeExcelController class. The problem was that the function would never stop counting. Columns with 1,000 items would be in the tens of thousands before I figured out what was going on and stopped the application. I don’t know what other issues may arise by having only one tab—finding those would be all but impossible to test given the sheer volume functions available in Excel—but at least I know the issue is there and the condition is easy to test. As to why the number of worksheets would interfere with such a seemingly simple function is buried somewhere in Microsoft.
The third test—no data connections—is proving to be much more difficult and had the same problems as the second. The problem testing for this arises when trying to decipher what Excel means by a data connection. Is this a connection in a pivot table or is this a named item, and what can be done about the connection if one is found? Connections, in general, are pretty well scattered throughout the API. Connections are managed (so far as I have found) in two places: the
pivotCache class, accessed through the
workbook class, and the
namedItem class, accessed through the
worksheet class. Documentation on both is just as scattered (and non-existent on the Mac). In my case, it appears the
namedItems array is where my found connections are held. In Applescript, there is no way to remove one once it has been created, and I haven’t gotten around to figuring out how to handle the existence of a named item. I am left with the same question as I do for the second condition: What does the existence of a named item have to do with getting information from an Excel spreadsheet? This is just weird.