Wednesday, June 02, 2010

Fixing -[NSMutableURLRequest setValue:forHTTPHeaderField:]

UPDATE

This problem is fixed as of Mac OS X 10.8.3+ and iOS 5.1+. I have no idea when it was actually fixed since radar sucks so much.

The problem

From NSMutableURLRequest setValue:forHTTPHeaderField: documentation:
In keeping with the HTTP RFC, HTTP header field names are case-insensitive.

I see three problems with this.
  1. Assuming all HTTP implementations are RFC compliant is foolish to say the least.
  2. Enforcing case-insensitivity is nonsense.
  3. This bit of documentation is accurate, the case of header fields is actually changed.
Trying this
    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:@"http://www.example.com"]];
[request setValue:@"MyValue" forHTTPHeaderField:@"MyField"];
NSLog(@"result: %@", [request allHTTPHeaderFields]);
outputs
    result: {
Myfield = MyValue;
}
Notice how the 'F' of MyField was lowercased, that sucks! If the HTTP server you are trying to communicate with is case-sensitive, you are screwed. I filed radar #8029516, which is a duplicate of radar #3131623, which means there is almost no chance to have this fixed anytime soon.

When this happens, you should contact the server administrator asking for a case-insensitive implementation. But in the meantime, if you badly need to preserve the case of your headers, read on.

The investigation

First, let's fire otx to disassemble the Foundation framework in order to have a look at setValue:forHTTPHeaderField: implementation. I strongly suggest you to use otx version from subversion trunk, as it is better at resolving symbols in tail call optimizations (fixed in r553).

Excerpt from otx 0.16b:
   +18  [...]  movl     %eax,0x08(%ebp)
+21 [...] leave
+22 [...] jmpl 0x002c52bb

Excerpt from otx trunk:
   +18  [...]  movl     %eax,0x08(%ebp)
+21 [...] leave
+22 [...] jmpl 0x002c52bb _CFURLRequestSetHTTPHeaderFieldValue

Much better, isn't it?

So let's go through the function calls of setValue:forHTTPHeaderField:. With a basic static analysis of the Foundation and CFNetwork disassemblies, we can draw the following call path:
-[NSMutableURLRequest(NSMutableHTTPURLRequest) setValue:forHTTPHeaderField:]
CFURLRequestSetHTTPHeaderFieldValue()
URLRequest::setHTTPHeaderFieldValue()
HTTPRequest::setHeaderFieldValue()
CFHTTPMessageSetHeaderFieldValue()
HTTPMessage::setHeaderFieldValue()
_CFCapitalizeHeader()

_CFCapitalizeHeader looks like a very good candidate for being the bastard changing the case of our headers. A quick search reveals the source code of CFNetwork that was open source long time ago. Although the open source implementation does not exactly match what we see in the disassembly (it is now using toupper instead of ch + 'A' - 'a' for example), we are now absolutely sure that _CFCapitalizeHeader is the function responsible for messing with our headers.

Now, let's check is if there is a path that will not call _CFCapitalizeHeader and if we can somehow influence the condition that would avoid the call to _CFCapitalizeHeader. This is quickly checked, especially if you enabled the Separate logical blocks option of otx (-b option for cli).
HTTPMessage::setHeaderFieldValue(__CFString const*, __CFString const*)
+0 000515a4 55 pushl %ebp
+1 000515a5 89e5 movl %esp,%ebp
+3 000515a7 83ec28 subl $0x28,%esp
+6 000515aa 8b4508 movl 0x08(%ebp),%eax
+9 000515ad 8975f8 movl %esi,0xf8(%ebp)
+12 000515b0 8b7510 movl 0x10(%ebp),%esi
+15 000515b3 897dfc movl %edi,0xfc(%ebp)
+18 000515b6 8945f4 movl %eax,0xf4(%ebp)
+21 000515b9 8b450c movl 0x0c(%ebp),%eax
+24 000515bc 890424 movl %eax,(%esp)
+27 000515bf e88a26fbff calll __CFCapitalizeHeader
+32 000515c4 c744240cffffffff movl $0xffffffff,0x0c(%esp)
+40 000515cc 89742408 movl %esi,0x08(%esp)
+44 000515d0 89c7 movl %eax,%edi

We see that there is no path that avoid the call to _CFCapitalizeHeader. So we are left with the last resort solution: patching _CFCapitalizeHeader. With APE Lite, function patching is very easy. You first use APEFindSymbol() to find the address of a non-exported symbol (i.e. __CFCapitalizeHeader), then APEPatchCreate() to replace a function implementation with your own, while still keeping a reference to the original implementation. On iPhone OS, you can use APE Lite+arm (my implementation of the APE Lite API using MobileSubstrate).

The solution

NSMutableURLRequest+CaseSensitive is a category on NSMutableURLRequest that adds these three methods:
   - (void) setAllHTTPHeaderFields:(NSDictionary *)headerFields caseSensitive:(BOOL)caseSensitive;
- (void) setValue:(NSString *)value forHTTPHeaderField:(NSString *)field caseSensitive:(BOOL)caseSensitive;
- (void) addValue:(NSString *)value forHTTPHeaderField:(NSString *)field caseSensitive:(BOOL)caseSensitive;

Just pass caseSensitive:YES for preserving the case of your header fields.

Warning: you SHOULD NOT use this in production code. But hey, HTTP implementations SHOULD be case-insensitive ;-)

11 comments:

Julio said...

Just to be sure, I CAN NOT submit and IPHONE APP using this PATCH because apple will reject it right? or CAN I?

Cédric Luthi said...

I don't know, I have not tried. Please report in the comments if your app gets rejected or not.

Unknown said...

Hello,

I know it's an old post but i decided to use your fix (my server is also case-sensitive).

But i get an error:

"Detected an attempt to call a symbol in system libraries that is not present on the iPhone:
open$UNIX2003 called from function APEFindSymbol

If you are encountering this problem running a simulator binary within gdb, make sure you 'set start-with-shell off' first."

Any help, please?

Also thanks for the post, very helpfull

Daniela

mobileliga said...

Jika kalian tidak ingin mengalamai kekalahan ketika bermain slot online. Sebaiknya kalian bermain di situs Slot 24 Jam seperti Mobileliga, untuk menghindari kekalahan secara beruntun. Jika kalian bermain di Mobileliga sebagai pemain baru kalian bisa mendapatkan banyak sekali keuntungan. Tentunya ini akan membuat kalian bisa memperoleh profit yang berlipat ganda. Jadi jangan ragu ayo segera bermain di Mobileliga.

hokiemas said...

Gunakanlah Situs Judi Slot Online seperti Hokiemas jika ingin mendapatkan profit yang melimpah. Karena hanya di Hokiemas kalian bisa menikmati berbagai promo menarik dengan sangat mudah. Hokiemas juga termasuk kedalam situs besar yang bisa memberikan kemudahan dalam melakukan deposite atuapun penarikan tanpa potongan.

sevenqq said...

Situ judi QQ seperti SevenQQ itu merupakan situs yang sangat bisa dipercaya untuk bermain dan bertaruh. Bukan hanya itu tapi situs ini juga bisa memberikan kemudahan kepada setiap pemainnya. Jadi kalian yang sedang mencari Situs Judi QQ Online sevenQQ bisa menjadi solusi terbaik untuk tempat kalian bermain.

BeruangQQ said...

Setiap permainan QQ yang dilakukan seharunya dimainkan di situs yang tepat. Penggunaan Situs Judi QQ yang tepat ini memang sangatlah penting. Dengan memakai situs QQ yang tepat, keuntungan besar bisa langsung diperoleh. Oleh karenanya, pilihlah selalu situs QQ yang terpercaya layaknya BeruangQQ untuk bertaruh.

ligamansion dua said...

Selama kalian bermain judi Slot Online apakah kalian pernah bermain dan mendapatkan profit yang besar? Jika kalian belum pernah sebaiknya kalian mencoba untuk bermain di Ligamansion2. Karena di situs ini selalu memberikan hadiah jackpot yang sangatlah besar sekali bahakan bisa mencapai Milyaran rupiah.

togel said...

Seberapa sulitkah menemukan Bandar Togel yang terpercaya itu? Jawabannya gampang - gampang susah. Namun jika kesulitan untuk menemukan situs yang tepat, ada baiknya untuk main aman saja dengan mendaftar ke Togel. Situs tersebut amat cocok digunakan para petaruh dan pemain judi togel online.

paluqq said...

Kemudahan dalam memilih situs Domino QQ Online yang terpercaya di Indonesia memang pasti terjamin kredibilitasnya. Pastikan untuk menggunakan situs yang tepat macam PaluQQ. Dengan menggunakan situs yang terpercaya seperti PaluQQ, keuntungan maksimal bisa diperoleh dengan sangat cepat dan mudah. Oleh sebab itu, gunakan terus PaluQQ.

Mochammad Raihan said...
This comment has been removed by the author.