I have a simpler way. You'll want Homebrew if you don't already have it:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install duti:

brew install duti

Now you need to find the id of the app you want to use, and assign it to the extension you want to use it for. In this example, I already use Brackets for *.sh and I want to also use it for *.md files instead of xcode.

Get the default app id for .sh files:

duti -x sh

output:
  Brackets.app
  /opt/homebrew-cask/Caskroom/brackets/1.6/Brackets.app
  io.brackets.appshell

The last line is the id.

Use this app id for all .md files:

duti -s io.brackets.appshell .md all

This information is stored in the file's resource fork (Wikipedia). These resource forks are exposed as extended attributes (Wikipedia):

$ ls -l@ somefile.txt 
-rw-r--r--  1 danielbeck  staff  0 18 Mär 19:00 somefile.txt

# setting non-default application using Finder

$ ls -l@ somefile.txt 
-rw-r--r--@ 1 danielbeck  staff  0 18 Mär 19:01 somefile.txt
        com.apple.ResourceFork  1338

Editing from scratch is probably quite painful -- lots of binary data:

$ xattr -p com.apple.ResourceFork somefile.txt 
00 00 01 00 00 00 05 08 00 00 04 08 00 00 00 32
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 04 04 00 00 00 32 2F 55 73 65 72 73 2F 64
61 6E 69 65 6C 62 65 63 6B 2F 41 70 70 6C 69 63
61 74 69 6F 6E 73 2F 53 75 62 6C 69 6D 65 20 54
65 78 74 20 32 2E 61 70 70 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 01 00 00 00 05 08
00 00 04 08 00 00 00 32 0C 00 00 00 C0 05 00 00
00 1C 00 32 00 00 75 73 72 6F 00 00 00 0A 00 00
FF FF 00 00 00 00 19 00 00 00

But you can treat these resource forks like files. For example:

$ open somefile.txt # opens in Sublime 2
$ cp somefile.txt/..namedfork/rsrc openInSublime2rsrc
$ open otherfile.txt # opens in TextEdit
$ cp openInSublime2rsrc otherfile.txt/..namedfork/rsrc
$ open otherfile.txt # opens in Sublime 2

/..namedfork/rsrc is how the resource fork is exposed to POSIX applications (i.e. probably everything you do in Terminal).

So you just need to create one "template" file from an existing resource fork (in this example openInSublime2rsrc), and you can copy it to your other files afterwards.

You have to associate you app with the file types you want it to open. You do this by adding some parameters to your Info.plist.

This post explains it: How do I associate file types with an iPhone application?

Maybe this thread can help regarding issue 2:

http://lists.apple.com/archives/installer-dev/2009/Sep/msg00036.html

My solution to the problem is:

1. Create a new application in Xcode that contains the icon resources for the file type.
2. Select that app as the default for the file type.
3. Substitute the app's binary inside its bundle for a hard link to the binary of the app you want to open the file.

EDIT: This only appears to work in Mojave: in Catalina, there's some code-signing fatal crash. If anyone know how to fix it...

Create a new app in Xcode

This step requires no coding whatsoever. Open a new Project as a Document-based application, using the file extension of the file type you want to change.

Under 'Assets.xcassets' in the sidebar, click on the + icon and under App Icons & Launch Images, select create a New MacOS Generic Icon.

Add the correctly sized images to the empty squares.

Next, in Targets > Info, define the Document Types fields with the relevant data for your file type. Select the Icon resource, and you should get the icon in the empty square.

You can do the same for Imported UTIs, just to cover all bases.

Your app should build and run without any further effort. Quit it. Back in Xcode, right-click on the app product and select "Show in Finder". Move the app to the Applications folder.

The LaunchServices database may not immediately update with your new app and its icons. One tip to force it to update is to copy and paste an icon into the icon square in Get Info on any file. You can delete the change immediately.

Set your app to be the default app for the file type.

This is done in the usual way in Get Info window. The Finder might not update existing windows: close and reopen to see the icon change.

Substitute the binary

Right-click on your new app. Select Show Package Contents. Navigate to the MacOS folder, and delete the Unix executable file inside, with the name of your app.

In another Finder window, navigate to the similar MacOS folder inside the bundle of the application that you want to open the file. (In my case, BBEdit.)

In the Terminal, type ln (that's lowercase LN) followed by a space. Then drag the Unix executable of the 'destination' app onto the Terminal window. Then drag the the empty Macos folder of your app onto the Terminal window. The completed command should look something like this:

ln /Applications/BBEdit.app/Contents/MacOS/BBEdit /Applications/PostScript.app/Contents/MacOS

Press Enter.

You should now have a hard link to the destination app (BBEdit) in your dummy app.

One important thing left to do: rename this link to the name of your app. (In my case, 'PostScript'.) You could do this in one step as part of the Terminal command above, by appending the name.

ln /Applications/BBEdit.app/Contents/MacOS/BBEdit /Applications/PostScript.app/Contents/MacOS/PostScript

Done.

As long as your dummy app and the destination app are both installed, files will display the custom icons, and launch with the app of your choice, without modifying installed apps. There doesn't seem to be any speed penalty, either.

Icons for file types are handled by Launch Services (the service determining, among other things, which application handles a file type). The file icon is always provided by the application handling opening a file by default. This way, e.g. Preview provides PDF icons by default, and PNG icons, but if you change all PNG files to open using Pixelmator instead, these files get a Pixelmator-style icon afterwards.

What you need to do is add your file type definition to the application that opens it by default, or create your own "dummy" application for the file type.

As an example, my system does not yet know about .scala, and I want TextMate to handle it.

First, I Show Package Contents of TextMate.app, navigate to Contents/ and open Info.plist, either with a text editor, or Property List Editor, part of Apple's developer tools.

TextMate uses an unusual format for Info.plist (it's usually binary or XML), an excerpt of which looks like this:

CFBundleDocumentTypes = (
    {   CFBundleTypeName = "ADA source";
        CFBundleTypeExtensions = (adb, ads); 
        CFBundleTypeIconFile = ADA; 
    },

The parent key CFBundleDocumentTypes is what we want. The first child element of it, enclosed in curly braces, contains a file type definition, complete with name (for file type column in Finder), file extensions, and the name of the icon file (ADA for TextMate.app/Contents/Resources/ADA.icns).

To support .scala, we need to add a sibling element similar to the one above, under the same parent element CFBundleDocumentTypes:

CFBundleDocumentTypes = (
    {   CFBundleTypeName = "ADA source";
        CFBundleTypeExtensions = (adb, ads); 
        CFBundleTypeIconFile = ADA; 
    },
    {   CFBundleTypeName = "Scala source";
        CFBundleTypeExtensions = (scala); 
        CFBundleTypeIconFile = ADA; 
    },

For simplicity's sake, I'm using the same icon file, but we can create one ourselves, copy it to TextMate.app/Contents/Resources/SCALA.icns and refer to it as SCALA in Info.plist.

Now, close TextMate if it's running, move the application TextMate.app to a different folder, and open it again. Close it, and move it back, then open it yet again. This is done so Launch Services picks up the changes to Info.plist we just performed.

The result looks like this (remember, we reused the TextMate Ada icon for .scala):

If you don't want to associate an application to open .scala files (can't imagine why, but there's a way to do this): Change the additions to the file to the following:

    {   CFBundleTypeName = "Scala source";
        CFBundleTypeExtensions = (scala); 
        CFBundleTypeIconFile = ADA; 
        CFBundleTypeRole = "None";
    },

Now, the application declares the file type, but tells the system that it does not know how to handle it (Apple gives the example of Finder declaring font types, although it cannot open them itself).

*Note that any permanent application assignment to the file type overrides the icon, probably to the "unknown document" kind.

If you already have an application associated with .scala, and just want to change the file icon without changing the associated application:

Open of the application's Info.plist, look for the file type entry in CFBundleDocumentTypes, and get the CFBundleTypeIconFile. Modify this file in .../Contents/Resources/ and it should be reflected in Finder shortly afterwards.

Bonus information:

How to change a single document's or folder's icon to an image file

Suppose you have an image ~/Desktop/test.png which you want to use an an icon (it's the screenshot used above to illustrate the .scala/TextMate/Ada icon change.

Open /Applications/Utilities/Terminal.app and enter:

sips -i ~/Desktop/test.png

This will change the test.png file icon to its image. Since I enjoy recursion, lets try this:

Before, the plain Preview png icon (image preview is deactivated):

After, the image file itself is its preview:

Now we can open the file's Get Info dialog, click the desired, future document icon on the top left, Cmd-C to copy it, open the document file's Get Info dialog, click the undesired, current document icon on the top left, and Cmd-V to paste the icon we want.

Alternatively, you can open the image file you want to use as icon in Preview, Cmd-A to select all, Cmd-C to copy the image to clipboard. Then open the target document's Get Info dialog, select the icon to change in the top left, and Cmd-V to paste the image over it.