View unanswered posts | View active topics It is currently Sep 7th, '10, 05:49




Post new topic Reply to topic  [ 1 post ] 
 Hook engine design.. 
Author Message
Site Moderator
User avatar

Joined: Aug 18th, '09, 03:32
Posts: 813
Post Hook engine design..
More like kind of a "blog post here"..

Been working on my own binary code and API hook/detour library.
I've made my own support routines to hook arbitrary code, etc., but for the most part used madCodeHook (http://www.madshi.net/madCodeHookDescription.htm) for API hooks and DLL load/injection.

It's very stable for API hooks, Madshi has gone to great lengths to make it very robust.
Including the smallest little details.
For one, most hook engines map in APIs as soon as you try to hook them.
Thus violating the native behavior of the process.
While madCodeHook only hooks API's if the DLL is already loaded or wait until it does (if ever).
Most of the time it probably doesn't matter but then it does it "right" anyhow.

But the problem with madCodeHook, MS Detours, and any other systems I know of is that they don't have real support for arbitrary code hooks.
They are built around function, mainly API hooks.

I haven't seen some sort of dynamic support to be able to just tap into a code spot some place and and have it auto-generate a trampoline/gate/stub for you.
I often do this, if not for some game hack, but just to dynamically monitor something when I do reversing.
I end up making my own little code stubs that are pretty time consuming, it would be nice if the code was generated automatically.
And it can get pretty complicated because the processor (registers, stack, and to lessor consideration FPU & SSE) are in a certain state. You can't just go put something in a register because it might be used.
What you want to do is save the current context, go do your hook, perhaps modifying the context as desired and return (with out stepping on something and crashing the process).

Also another problem madCodeHook and any other system I've seen in detail only support a sort of a "sub-classing" concept. This is great in principle, works great for standard API calls, easy to setup, etc.
But has little or no consideration for other calling conventions other then "_stdcall" and "__cdecl".
If a function like a C++ method call with a passed along "this" (usually via the ECX register) pointer there is no guarantee the hook engine won't step on ECX. Let alone you don't know if your own hook function steps on it, and there is no direct way to read it.
If you are lucky and some how the hook engine doesn't step on ECX one can make a hook stub (or the whole hook) in assembly for it.
The same goes for "__fastcall" conventon if not even worst as arguments often passed in EAX and EDX.
And complicated more with "Link time code generation" where any number of registered can be used and carried around through multiple function calls.
Conventional "sub-class" setups just choke, at best unpredictable in these cases.

I hope to solve some of these problems with my own engine, and add some unqiue features to support HWBP (Hardware break point) hook and other techniques transparently as possible.


Mar 6th, '10, 08:04
Profile
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 1 post ] 


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Based on design by Copyright© 2009 Vjacheslav Trushkin, modded by Sirmabus Copyright© 2009