Signing a raw transaction with Python ECDSA (or OpenSSL)

Question

I'm looking for specifics of Step15-17 from Redeeming a raw Tx Step By Step, which is essentially the step where the concatenated raw Tx structure is double sha256 hashed, and then signed with an ECDSA library. I've also referred to other raw transactions guides; I'm looking specifically how to sign a raw hex Tx structure without a GUI/website/etc

I've constructed a testnet Tx using Bitcoincore v0.10 (a full node) and attempted the same with Python and cannot correctly sign the raw Tx structure using Python ecdsa.

• TxID: 72b764383b99fb3d112ac8b474a5d7c4242b75dbfee2d4e9cf9a6703d90f805a
• vout: 1
• To: n1hjyVvYQPQtejJcANd5ZJM5rmxHCCgWL7
• Amount: 0.990 BTC

I'll show the Bitcoincore commands first:

importprivkey 93FxXUeMJp93YQAtGeW5cE23gFN4sJbBr1RBmerLFVUDuqQqKL5
mu858WTEPiWWpAJRTMxC4ka6DJqiaCZiSB
createrawtransaction '[{"txid" : "72b764383b99fb3d112ac8b474a5d7c4242b75dbfee2d4e9cf9a6703d90f805a", "vout" : 1}]' '{"n1hjyVvYQPQtejJcANd5ZJM5rmxHCCgWL7" : 0.990}'
signrawtransaction 01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b7720000000000ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

Final bitcoincore hex Tx (using signrawtransaction): 01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000008a47304402204d78d2e6c0f801573e4960fb8e51ad939380d119d25f97d15efdedf815b05f02022066bd2ab0b401e32e7ce67ea45f8224097eeafbef2335d563776e5efe6632732d01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

Now the Python 2.7.9 code follows (FYI, using iPython 2.0 so there's boilerplate missing in the following code) :

# please excuse the hexlify/unhexlify, I work better in strings than bytes
import __future__
from pybitcointools import sha256
import hashlib, ecdsa
from ecdsa import SigningKey, SECP256k1
addr, pubkey = 'mu858WTEPiWWpAJRTMxC4ka6DJqiaCZiSB', '0479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1'
rawtx = '01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b7720100000000ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000' # TX structure with 00 as scriptSig, from bitcoincore createrawtransaction
# replacing the '00' scriptSig value
unsigned.append(rawtx[:82])  
unsigned.append('19'+'76a914953de657be4b305f606d9a9fbd35b070a682475788ac') # scriptSig = scriptPubKey input
unsigned.append(rawtx[84:]) 
unsigned.append('01000000')  # appending sighash_all
unsigned = ''.join(unsigned)

unsigned = '01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000001976a914953de657be4b305f606d9a9fbd35b070a682475788acffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac0000000001000000'

# See https://github.com/warner/python-ecdsa/blob/master/README.md
sig1 = sk.sign(sha256(sha256(unsigned)), hashfunc=hashlib.sha256, sigencode=ecdsa.util.sigencode_der) # note this references 2 different sha256 functions
sig2 = [hex(len(sig1 + '\x01'))[2:], hexlify(sig1), hexlify('\x01'), hex(len(unhexlify(pubkey)))[2:], pubkey]
sig2
>>> ['48',  '304502200e98d54ad642488121fa1fd4d055ff8f5b40773a21ecb42b5ef44ed9fd3b103c022100a2a56ffc3f27021a4be5d1d7a3016eff08fe7f550a4a894cca845ceab54dc53b', '01', '41', '0479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1']
sig2 = len(''.join(map(unhexlify, sig2)))
hex(len(''.join(map(unhexlify, sig2))))[2:] 
>>> '8b'  # push 139 bytes
sig3 
>>> 8b48304502200e98d54ad642488121fa1fd4d055ff8f5b40773a21ecb42b5ef44ed9fd3b103c022100a2a56ffc3f27021a4be5d1d7a3016eff08fe7f550a4a894cca845ceab54dc53b01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1 
signed = unsigned[:82] + sig3 + unsigned[84:]  # inserting the signature into the 00 the core software uses in place of ScriptSig

Final (invalid) Python Tx:

01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000008b48304502200e98d54ad642488121fa1fd4d055ff8f5b40773a21ecb42b5ef44ed9fd3b103c022100a2a56ffc3f27021a4be5d1d7a3016eff08fe7f550a4a894cca845ceab54dc53b01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000

Sending the Python generated Tx does not work. If I sendrawtransaction <PYTHON HEX> in Bitcoincore it returns an invalid signature error (Specifically, 16: mandatory-script-verify-flag-failed (Script evaluated without error but finished with a false/empty top stack element) (code -26)).

(TL;DR) QUESTION: Where am I going wrong in signing the raw transaction structure manually?

NB: I am aware pybitcointools and related libraries can sign Txs, but I am looking for specifics, as even these two very thorough Bitcoin SE resources (LINK1, LINK2) brush past the details. I'd prefer an answer for Python ECDSA or PyCrypto, however OpenSSL I suppose is the next best answer.

Answer 1

OK, I figured out how to sign the raw Tx using Python ecdsa. I'll step through it:

Recall:

createrawtransaction '[{"txid" : "72b764383b99fb3d112ac8b474a5d7c4242b75dbfee2d4e9cf9a6703d90f805a", "vout" : 1}]' '{"n2kx7k6JuA5Wy27fawaeiPX7dq8mbRDPAv" : 0.99}' = 01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b7720100000000ffffffff01c09ee605000000001976a914e900510876cb689f1db6fa982376c301362b740c88ac00000000

And then adding in the scriptPubKey for scriptSig's 00:

unsigned = '01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000001976a914953de657be4b305f606d9a9fbd35b070a682475788acffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac0000000001000000'

Note unsigned has the 01000000 appended because the SIGNHASH is 01 (which will also be appended to the DER signature in a minute)

So, we double SHA256 bytes and return txhash as bytes: txhash = hashlib.sha256(hashlib.sha256(unsigned.decode('hex')).digest()).digest()

We also need the privkey for mu858WTEPiWWpAJRTMxC4ka6DJqiaCZiSB (in secret exponent form, ie hex): privkey = 'dc57c6d067376c36bbed632c9d00f03767867f337d5a86b5b0308a60004f08ee'.zfill(64) privkey = 'dc57c6d067376c36bbed632c9d00f03767867f337d5a86b5b0308a60004f08ee'

Now, we sign using this corrected code:

signingkey = ecdsa.SigningKey.from_string(privkey.decode('hex'), curve=ecdsa.SECP256k1)
SIG = signingkey.sign_digest(txhash, sigencode=ecdsa.util.sigencode_der) +'01'.decode('hex')

This is working code. From here on in, we're simply making the signature:

ScriptSig = <varint of total sig length> <SIG from code, including appended 01 SIGNHASH> <length of pubkey (0x21 or 0x41)> <pubkey>

where pubkey = '0479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1

Giving us a valid signed Tx without using bitcoincore:

01000000015a800fd903679acfe9d4e2fedb752b24c4d7a574b4c82a113dfb993b3864b772010000008a47304402204d78d2e6c0f801573e4960fb8e51ad939380d119d25f97d15efdedf815b05f02022066bd2ab0b401e32e7ce67ea45f8224097eeafbef2335d563776e5efe6632732d01410479b22a5127d176a49d506c86f791031f94a389227ef46a8ddb725a88c944c37e3f753de6ee0b441a0237801f140810e111a1fd8276a2a5d0ee07224a1b551cc1ffffffff01c09ee605000000001976a914dd6cce9f255a8cc17bda8ba0373df8e861cb866e88ac00000000