Category Archives: Apple

Mac OS X Mojave and SSH-AGENT notifications

For work (this time around), I have been given a new Macbook Pro 12″ running Mac OS X version 10.14.1, also known as Mojave.

I was missing my ssh-agent notifications on this new mac (see links below), so it’s time to replay my recipe wherein I patch Apple’s OpenSSH to integrate support for Notification Center.

Original article:
http://devinteske.com/wp/ssh-agent-notifications-osx/
Update to original:
http://devinteske.com/wp/replay-mac-os-x-and-native-ssh-agent-notifications/
Further update to original:
http://devinteske.com/wp/mac-os-x-el-capitan-and-native-ssh-agent-notifications/

However, things have changed slightly in Mojave, so we’ll explore a new set of instructions to simplify the process.

How to patch Apple’s OpenSSH on Mojave to support native Notification Center in 17 steps (Mojave specific changes highlighted in red):

REMINDER: You can browse Apple’s open source repository at https://opensource.apple.com/

  1. Open Terminal.app
  2. Mac OS X 10.14.1:
    curl -LO https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.2.7.tar.gz
    Mac OS X 10.14.3:
    curl -LO https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.6.5.tar.gz
  3. tar zxf !$:t
  4. INC=`pwd`/!$:r:r/include
  5. curl -LO https://opensource.apple.com/tarballs/OpenSSH/OpenSSH-220.220.4.tar.gz
  6. tar zxf !$:t
  7. cd !$:r:r/openssh
  8. curl -L https://github.com/devinteske/apple/commit/5063bab155cdcd5d160d94aea0a7a09fd9999896.patch -o patch.txt
  9. patch -N < !$
  10. CC=clang CFLAGS="-I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I$INC" LDFLAGS="-isysroot / -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/lib" ./configure --with-pam --with-audit=bsm
  11. make TARGETS=ssh-agent
  12. Disable System Integrity Protection using Apple SIP Guide
  13. sudo cp -avf /usr/bin/ssh-agent{,.orig}
    Note: This step requires SIP to be disabled
  14. sudo cp ssh-agent /usr/bin/ssh-agent
    Note: This step requires SIP to be disabled
  15. Optional: Re-enable SIP using Apple SIP Guide
  16. killall ssh-agent
  17. ssh some-host
    Note: If you’re just using ~/.ssh/id_rsa and haven’t added the key to your keychain, no notification will appear and you will have to execute “ssh-add -K ~/.ssh/id_rsa” (without quotes; make sure to use capital-K) to add the key to your keychain.

Now every subsequent ssh request that uses a key stored in your keychain will cause a notification to appear in Apple’s built-in Notification Center. Cheers!

Mac OS X El Capitan and Native ssh-agent Notifications

For work, I have been given a new Macbook Pro 13″ running Mac OS X version 10.11.6, also known as El Capitan.

I was missing my ssh-agent notifications on this new mac (see links below), so it’s time to replay my recipe wherein I patch Apple’s OpenSSH to integrate support for Notification Center.

Original article:
http://devinteske.com/wp/ssh-agent-notifications-osx/
Update to original:
http://devinteske.com/wp/replay-mac-os-x-and-native-ssh-agent-notifications/

However, things have changed slightly in El Capitan, so we’ll explore a new set of instructions to simplify the process.

How to patch Apple’s OpenSSH on El Capitan to support native Notification Center in 16 steps (El Capitan specific changes highlighted in red):

REMINDER: You can browse Apple’s open source repository at https://opensource.apple.com/

  1. Disable System Integrity Protection using Apple SIP Guide
  2. Open Terminal.app
  3. curl -LO https://opensource.apple.com/tarballs/OpenSSL098/OpenSSL098-59.60.1.tar.gz
  4. tar zxf !$:t
  5. sudo cp -R !$:r:r/src/include/openssl /usr/local/include/
  6. curl -LO https://opensource.apple.com/tarballs/OpenSSH/OpenSSH-195.40.1.tar.gz
  7. tar zxf !$:t
  8. cd !$:r:r/openssh
  9. curl -L https://github.com/devinteske/apple/commit/f13a501678ed79bb9511828423e754df344e3bf0.patch -o patch.txt
  10. patch -N < !$
  11. ./configure --with-pam --with-audit=bsm
  12. make
  13. sudo cp -avf /usr/bin/ssh-agent{,.orig}
    Note: This step requires SIP to be disabled
  14. sudo cp ssh-agent /usr/bin/ssh-agent
    Note: This step requires SIP to be disabled
  15. killall ssh-agent
  16. ssh some-host
    Note: If you’re just using ~/.ssh/id_rsa and haven’t added the key to your keychain, no notification will appear and you will have to execute “ssh-add -K ~/.ssh/id_rsa” (without quotes; make sure to use capital-K) to add the key to your keychain.
  17. Optional: Re-enable SIP using Apple SIP Guide

Now every subsequent ssh request that uses a key stored in your keychain will cause a notification to appear in Apple’s built-in Notification Center. Cheers!

Write Images to USB with AppleScript

As the only release engineer at $work, I have been tasked with creating the bootable USB devices of our FreeBSD-based product to send to the Contract Manufacturer. Every time a new release made it to General Availability (GA) status, Sabine would drop off a couple USB sticks and tell me which build to load onto them.

Wanting to empower her, I set out to create a better approach; one that did not involve dropping to the command-line and tapping-out a “dd” command. Meet a new AppleScript, named “WriteImage” written to take the guess-work out of the process.

http://druidbsd.cvs.sf.net/viewvc/druidbsd/WriteImage/

With WriteImage you can now take the USB image file (*.img or *.img.gz) and drop it on the AppleScript droplet. It automates the process of generating the necessary commands to image the file onto hardware USB. Tell the app through a series of clicks or drag/drop actions which image to write, it then prompts you to insert a USB device, and then asks for confirmation (twice) before launching Terminal to do the dirty work.

Let’s see it in action (photo tour):

Selecting an Image file for WriteImage
WriteImage initial dialog when double-clicked.
Prompt to insert USB device
WriteImage prompt, waiting for you to insert a USB device to image.
Last chance before proceeding
WriteImage displays a Last Chance dialog before proceeding to do anything.
Writing the Image to USB
WriteImage runs a shell script that does the dirty work.
Imaging the USB Drive
WriteImage ultimately results in an automatically crafted “dd” command run via Terminal
Get progress with Ctrl-T
When “dd” is running, press Ctrl-T on the keyboard to see how much data has been written and how fast
Success
As-is the case with FreeBSD images, when the process is complete, your Mac will give this error. Success in disguise.
Initializing a Drive
If you need to image a drive that is not currently readable by Mac OS X, click “Initialize…” and use Erase

Cheers!

REPLAY: Mac OS X and Native SSH-Agent Notifications

Having recently updated my 2014 Macbook Air 11″ from Mac OS X 10.10 to 10.10.4, I lost my customizations to Apple’s OpenSSH and no longer was I receiving anything in the Notification Center for SSH logins using my keychain. Looks like it’s time to rehash the steps necessary to reload support for the Notification Center for those going through the same ordeal.

For reference here’s the original blog post where I introduced the customizations required to get Apple Notification Center support for OpenSSH on Mac OS X:

http://devinteske.com/wp/ssh-agent-notifications-osx/

Step-by-step we’ll go through the motions of re-obtaining the latest copy of Apple’s modified OpenSSH, to be patched.

NOTE: Make sure you’ve updated Xcode in Apple Software Update, else you’ll get the error "configure: error: Your OpenSSL headers do not match your library" during compilation steps below.

ASIDE: You can browse Apple’s opensource repository at https://opensource.apple.com/ (obtaining the latest copy of OpenSSH with Apple’s customizations couldn’t be easier). Rather than documenting how to navigate said page, below steps will instead use direct links to the software we’re recompiling (for brevity).

How to patch Apple’s OpenSSH to support native Mac OS X Notification Center (in 13 easy steps):

  1. Open Terminal.app
  2. curl -LO https://opensource.apple.com/tarballs/OpenSSH/OpenSSH-189.tar.gz
  3. tar zxf !$:t
  4. cd !$:r:r/openssh
  5. curl -L https://github.com/devinteske/apple/commit/296d954851dbba2384797620c1b9a77e562917b8.patch -o patch.txt
  6. patch -N < !$
  7. ./configure --with-pam --with-audit=bsm
  8. make
  9. sudo cp -avf /usr/bin/ssh-agent{,.orig}
  10. sudo cp ssh-agent /usr/bin/ssh-agent
  11. killall ssh-agent
  12. ssh some-host
  13. Click "Always Allow" to confirm access from new ssh-agent to Keychain

Now every subsequent ssh request will give you a notification in Apple's built-in Notification Center. Cheers!

Mac OS X and Native ssh-agent Notifications

Apple Mac OS X makes working with SSH private keys easy and convenient. Things that you can safely forget and drop from your routine when using Mac OS X are:

  • How and when to launch ssh-agent(1)
  • Continuously loading your private key

Apple has integrated launchd(8) and Keychain support into ssh-agent(1) so it launches automatically on-demand and can interact with keys stored in Keychain Access.

First, a brief look at the above integration before we go improving things using new features introduced in Mac OS X 10.8 and higher.

EDIT: If you’re reading this thinking “Mac OS X has had Keychain and launchd integration for a while,” skip on ahead to “That being said…” below).

Traditionally when using an agent you would first manually launch an instance of ssh-agent(1) and then follow that up with:

ssh-add ~/.ssh/id_dsa

while on Mac OS X there is:

ssh-add -K ~/.ssh/id_dsa

which both loads your key into the running ssh-agent(1) and imports it into your Keychain. So you’ll find that quite untraditionally if you kill the running agent and then try to ssh(1) to a remote host, a new agent is launched on-demand and it loads the private key through the Keychain (login succeeds despite conventional logic about agents).

In the following picture, we can see previously loaded key(s) if we filter on the word “ssh” in Apple’s Keychain Access application (provided in the Utilities folder accessible via Command-Shift-U keyboard shortcut while in the Finder application):

View SSH private keys loaded with `ssh-add -K' in Keychain Access
View SSH private keys loaded with “ssh-add -K” in Keychain Access

If you delete this key from Keychain Access application then ssh(1) logins requiring that key no longer succeed as the running agent immediately loses access to said key. So the Keychain can be thought of as a persistent store for the agent; one that is tied to your Mac’s login account.

The automatic re-launching of ssh-agent(1) through launchd(8) means that if you kill the running agent it will be re-launched not immediately but only when access is attempted to a launchd(8)-provided default $SSH_AUTH_SOCK value as-configured in /System/Library/LaunchAgents/org.openbsd.ssh-agent.plist.

Initially when Mac OS X boots, ssh-agent(1) is not started. It is only when you perform your first ssh(1) (or related) command that launchd(8) invokes the following command:

/usr/bin/ssh-agent -l

You can see launchd(8)‘s integration in-action by opening a Terminal immediately after booting your Mac and first executing:

ps axwww | grep ssh-agent

You’ll see there are no instances of ssh-agent(1), yet. Meanwhile, if you execute:

echo $SSH_AUTH_SOCK

You’ll see launchd(8) provided a default value (similar to the following):

/tmp/launch-oLGVUi/Listeners

When any ssh(1) command attempts to access this launchd(8) provided $SSH_AUTH_SOCK listener, launchd(8) invokes the previously mentioned “ssh-agent -l” instance which then accesses private keys stored in your keychain. For example, assuming Remote Login is enabled in the Sharing section of System Preferences (which enables sshd), execute:

ssh localhost

After which, re-execute:

ps axwww | grep ssh-agent

To find a running instance of “/usr/bin/ssh-agent -l” (the “-l” indicating it was launched on-demand by launchd(8)).

This is all very slick integration that quintessentially eases your SSH life. In a nut-shell, it’s private keys provided from your Keychain on-the-fly whenever you use an ssh utility.

 


 

That being said, we can certainly improve upon this integration by adding new code to Apple’s version of OpenSSH. Primarily, since ssh-agent(1) can now be loaded on-demand and access data in my Keychain, I would like to get a notification every time someone uses the running agent (for security reasons and also coolness factor).

In Mac OS X Mountain Lion (10.8) and Mavericks (10.9) we are blessed with the new Notification Center API (http://support.apple.com/kb/ht5362). This built-in functionality replaces the need for 3rd-party software such as Growl (http://growl.info) required to perform notifications in previous releases of Mac OS X (10.7 and older).

RESOURCE: Apple’s custom version of OpenSSH (available from http://opensource.apple.com — specifically for Mavericks http://opensource.apple.com/tarballs/OpenSSH/OpenSSH-186.tar.gz).

I first tested integration of the NSNotificationCenter API on a non-Apple version of OpenSSH after failing to get my friend’s Growl-based notification patches to work (I didn’t want to pay $3.99 for Growl in the Mac App Store). Prior to Mac OS X 10.8, my friend @kang had produced the following work for getting notifications out of ssh-agent(1):

https://www.insecure.ws/2013/09/25/ssh-agent-notification/

EDIT: My friend just made a patched version of Simon G. Tatham’s Pageant that provides native Windows notifications — http://www.twitpic.com/e2v52x/full — available in both Source (https://github.com/gdestuynder/putty-pagent-notification) and binary release (https://github.com/gdestuynder/putty-pagent-notification/releases).

However, my good friend’s work was on the non-Apple version of OpenSSH and thus my resulting agent binary lacked launchd(8) integration and Keychain support. This meant that if I were to replace the existing /usr/bin/ssh-agent binary in Mavericks with the patched non-Apple version, I would lose those features previously discussed above, let alone that his patches were for an external notification system versus the new built-in Notification Center.

So I set out to build a drop-in replacement for Mavericks’ /usr/bin/ssh-agent carrying all the integration of the original.

EDIT: Keep reading toward the end for GitHub source link and link to Mavericks binary.

After unpacking Apple’s OpenSSH-186.tar.gz (link above) and applying my patches to add support for NSNotificationCenter, I quickly learned that some additional non-standard configure arguments were required.

Within the top-level openssh directory (of the unpacked OpenSSH-186.tar.gz archive) I executed the following to produce a binary truly worthy of replacing the stock /usr/bin/ssh-agent:

./configure --with-pam --with-audit=bsm
make

After which I had a patched ssh-agent binary in the current working directory, suitable for doing the following:

sudo cp -avn /usr/bin/ssh-agent{,.orig}
sudo cp ssh-agent /usr/bin/ssh-agent
killall ssh-agent

The very next ssh(1) command I execute causes a new fresh instance of my patched ssh-agent(1) to be invoked. The first time my patched agent attempts to access my Keychain, I get the following audit request (image below):

Granting patched ssh-agent(1) access to Keychain for the first time.
Granting patched ssh-agent(1) access to Keychain for the first time.

I choose the “Always Allow” option, knowing that if I want to change it later I can locate the private key in Keychain Access (HINT: Filter on “ssh”) and press the Command-I keyboard shortcut to edit Access Controls for that key.

My patched agent is programmed to hook into the new Notification Center and send you a notification when signing requests (image below).

Notification Center message from ssh-agent(1)
Notification Center message from ssh-agent(1)

Now every time I authenticate by way of private key, regardless of whether that key was loaded once with “ssh-add” or from the Keychain imported via “ssh-add -K“, I get a message from the Notification Center.

EDIT: The primary concern that inspired this work is when you enable agent forwarding to remote hosts. Logging into a compromised host with agent forwarding, a hacker can use your agent to log into yet uncompromised systems to which you have access via your agent. While notifications won’t prevent the use of your agent to reach new machines should you enable forwarding through a compromised system, the notifications will let you know when to (a) shut down your agent (b) take inventory of active connections where agent forwarding was enabled and (c) cull the logs for machines you were forwarding through in an effort to find suspicious activity indicating either a malicious hacker or black-hat administrator on the system. For a corporate laptop that primarily only ever connects to corporate networks (where such activity is never to be expected), a notification is more than adequate to quell the problem (versus a public network where it may be more advantageous to use an ask-pass utility to confirm key usage rather than notify after open usage).

How those messages appear are configured in the System Preferences for Notifications (easily accessible via the gear icon at bottom-right of the Notification Center Drawer). Only after you’ve received your initial message from the patched ssh-agent(1) can you then configure the specifics of how its messages appear. The patched agent pretends to be Keychain Access when talking to the Notification Center, so configuring the style and attributes for Keychain Access notifications will allow you to customize how these agent notifications appear.

As notifications are delivered, they stack up in the pull-out drawer of the Notification Center. I like to clear the log of ssh-agent(1) notifications before I go to bed and check for any when I wake up (perhaps indicating that my agent signed a login request when I wasn’t looking).

By default, I’ve made the agent play the “Submarine” sound during notifications. This can be changed by making your own build from the source code I’ve branched into Git offered below:

https://github.com/devinteske/apple/tree/master/OpenSSH-186/openssh

But you don’t have to modify the code if all you want to do is disable the notification sound. Within the Notifications section of System Preferences you can state that you don’t want the Keychain Access notifications to play a sound.

Last, but not least, I have made a precompiled binary with my modifications so you can simply download and copy into place (using the below command-line instructions):

curl -OL http://druidbsd.sf.net/download/ssh-agent+notifications.osx-10.9.2.tbz
tar jxf !$:t
sudo cp -avn /usr/bin/ssh-agent{,.orig}
sudo cp -fv ssh-agent /usr/bin/
killall ssh-agent

Executing these 5 commands and then logging into a remote machine with ssh(1) using private keys should generate a notification from Keychain Access (pictured above). Cheers!

If you need to rollback to the original ssh-agent for any reason, the above steps made a backup that can be restored typing the following commands:

sudo mv -f /usr/bin/ssh-agent{.orig,}
killall ssh-agent

Wherein your next ssh(1) command will cause the original ssh-agent(1) to be loaded (which doesn’t support NSNotificationCenter API).

XCode Update Woes – With Solution!

I’ve found that when you upgrade versions of XCode, that you lose the ability to compile against older SDK’s (and therefore, the ability to make a binary targeted for the older platforms). This is further annoying by the fact that every sample project that you download from Apple that is slightly older (by that I mean 6 months), you have to go in and “Edit Project Settings” to change the SDK from “4.0 Missing” to “iOS 4.3” before you can build/compile, install to device, or even run the App (I’m not complaining — a missing SDK is surely something worthy in commanding such manual intervention but — If you’re loading up over 200 sample apps from Apple, it gets quite tedious having to modify each project before being able to install it to the device).

I found the solution to all this, and you’ll be surprised at how much work Apple makes us do in order to maintain backward compatibility. This problem had me on the ropes for 2 weeks while I tried to figure out how to — with *either* the new XCode 4 or the still-offered legacy XCode 3.2.6 (both released last month) — compile ANY code for my iPhone 3G which — due to Apple’s decisioning — cannot update to the latest iOS. Without the below solution, I would not be able to compile code for the iPhone 3G without downgrading XCode. However, with the below solution, I can use the latest and greatest XCode with the older SDKs to compile code for the older devices.

Step 1: Stay up to date, but make backups!

Whenever Apple issues a new XCode installer, that installer only comes with the SDKs that they want to provide you. Currently, that’s only iOS 4.3 (note: the Device SDK is different than the Simulator SDK). Running said installer will wipe-out any existing old SDKs, leaving you only with what they have provided you in the latest installer. This is certainly cause for alarm, because if your device doesn’t use one of the provided SDKs, you cannot publish for that device. However, there are no rules for the AppStore that says you must baseline your app to a specific SDK, so whatever baseline you want to choose is fine with Apple — as long as it runs smoothly on the Device SDKs that Apple will test your App against.

Naturally, you’ll want to prevent the loss of any existing older SDKs before you install. Go to the Finder (by clicking the “Finder” icon in the Dock, or clicking the desktop background image, for example), and then select from the menubar Go->Go To Folder… (or simply press Apple+Shift+G). This will bring up a dialog prompt allowing you to go to a directory based on UNIX pathname. Enter the text “/Developer/Platforms/iPhoneOS.platform/Developer/SDKs” (without quotes) and press ENTER (aka return). This will open the directory where the SDKs are installed. Before installing the latest XCode Developer Tools from Apple, backup everything in this directory.

Step 2: If you want to install additional SDKs (one’s that you don’t have).

Visit this webpage to find links to the old Developer Tools.

NOTE: We’re not going to be installing these developer tools, but rather we’ll rip-open the images and pluck out only the SDK information (not XCode and associated developer tools such as gcc and friends).

http://iphonesdkdev.blogspot.com/2010/04/old-versions-of-iphone-sdk.html

Once you’ve downloaded the SDK, it should automatically mount on your desktop (if you’re using Safari anyway) as a disk image named “Xcode and iPhone SDK”. Within this disk image is a hidden directory named “Packages”. Since it is hidden, the simplest way to access this directory is to once-again use the Go->Go To Folder… menubar item (or press Apple+Shift+G), entering “/Volumes/Xcode and iPhone SDK/Packages” (without quotes) and pressing ENTER. This will allow you to browse the hidden “Packages” directory on the read-only disk image containing the Developer Tools. Within this directory are files named similar to:

iPhoneSDK3_0.pkg
iPhoneSDK3_1_2.pkg
iPhoneSDK3_1_3.pkg
iPhoneSDK3_1.pkg
iPhoneSDK3_2.pkg
iPhoneSDK4_0.pkg

…ad nauseum. Go ahead and double-click any of these “*.pkg” files to launch the SDK installer (which will install ONLY to the directory “/Developer/Platforms/iPhoneOS.platform/Developer/SDKs”.

As you launch the Installer, be aware that we’ll have to make some adjustments to allow the installation to occur. When prompted to select the target installation disk (immediately after clicking “Continue” on the initial dialog) click the hard disk that you want to install to (even if it’s already highlighted) to cause a “Choose Folder…” button to appear. Click the “Choose Folder…” button and pick somewhere that is writable as you (if you attempt to install to the default location, the installation will fail — because the installer did not pre-authenticate with root privileges as is done in the main-installer opposed to these sub-installers). I chose to make a new directory on my Desktop. After the installation is complete, you should have a “Developer/Platforms/iPhoneOS.platform/Developer/SDKs/{version}” directory in the target installation directory that you chose. This “{version}” directory will need to be plucked from where you installed it and dropped into the “/Developer/Platforms/iPhoneOS.platform/SDKs/” directory.

Once you’ve installed a new SDK, you’ll need to restart XCode.

Immediately after installing the device SDK required by any Apple Sample code and restarting XCode, you’ll then be able to simply select “Device” from the arch menu (at the top-left of the project window), and press Apple+R to run the app on your connected device. No more need to go into the project settings to change the default SDK (sometimes needs to be done for each sub-target) to one that exists.

Happy coding,
Devin Teske

Apple XCode 4 loses CVS support — D’Oh!

I felt that this was worth a blog-post.

So here we are. XCode (Apple’s interactive development system for building applications) version 4 has been released to the general public. I am wonderfully pleased with the simplification/unification efforts but XCode4 has me scratching my head why Apple would remove CVS support.

I look around me and all I see is CVS — sourceforge.net still firmly embraces CVS. Is this a sign that Apple doesn’t like open-source ventures such as sourceforge.net? Was maintaining CVS support too difficult? Were there some capabilities that were lacking in CVS?

As a person well entrenched in the ways of CVS, I find it ineffable that I might have to migrate sizable iOS projects to some other SCM system such as Subversion (SVN) or Git just to maintain GUI integration within XCode. Mind you, I have no qualms with doing all SCM from the command-line and leaving XCode as a debugger/profiler/analyzer/compiler/editor/builder (though I have to admit, having CVS SCM built-in to XCode 3 was oh-so nice — in-fact I use to pitch it to co-workers as a “does everything” editor that could even be used for unix projects too, optionally replacing a system of Makefiles).

I can only hope and pray that in the coming weeks/months, there is an overwhelming public outcry over the removal of CVS from XCode 4 and that Apple — hopefully being the kind-hearted and in-touch kind of people I could only imagine them to be — might resurrect this industry-standard (truly a fallen comrade). Until then, back to the command-line.

admin -ko,
Devin

P.S. If Apple decides to subsequently remove the command-line utility from the Developer Tools installation, I’ll be turning to MacPorts, Fink, or compile it from source (yeah, I’m that determined to use CVS).

P.P.S. With tools like gource [ http://code.google.com/p/gource/ ], nobody in their right mind would switch from their current SCM to any other unless their 15+ year commit history was guaranteed to be preserved in the conversion process.