1. Post #41
    mmavipc's Avatar
    February 2009
    812 Posts
    Everything starts from nothing. SteamRE is just the culmination of years of work and research done by us and others.

    Our Steam2 documentation can be traced back to work done by a team who developed a tool called the "Universal Content Launcher".

    You can see the similarities of the networking here: http://tools.assembla.com/svn/ucl/tr...ServerClients/.

    Their efforts can most likely be traced back to a person named "steamCooker" from -snip- who did a vast amount of research into the Steam2 servers and how they work. I don't know exactly who else to credit for the majority of the Steam2 networking being figured out, as the parts that are required for Steam2 logon were already well researched before we started work.

    The Steam3 documentation is a combination of two things:

    A very long time ago Valve accidentally released a debug build of their steamclient library which handles all Steam3 networking. Through this debug build we managed to extract structures and data definitions that helped pave the way to figuring out the UDP protocol.

    The rest of the information we have gathered is simply the work of long months of research, disassembly, and packet analysis. Knowledge of the API that is exposed by the steamclient library also helped us understand some of the design behind the networking.
    Might want to -snip- our favorite russian counter strike source site out of your post
    Reply With Quote Edit / Delete Reply United States Show Events Dumb Dumb x 10Disagree Disagree x 1 (list)

  2. Post #42
    deloc's Avatar
    January 2008
    1,125 Posts
    Might want to -snip- our favorite russian counter strike source site out of your post
    might want to expel yourself from this thread.
    Reply With Quote Edit / Delete Reply Canada Show Events Agree Agree x 27Zing Zing x 1 (list)

  3. Post #43
    Gold Member
    ShaRose's Avatar
    April 2007
    1,231 Posts
    Might want to -snip- our favorite russian counter strike source site out of your post
    Why, it's not like it's a big secret who they are.
    Reply With Quote Edit / Delete Reply Canada Show Events Agree Agree x 4 (list)

  4. Post #44
    Gold Member
    Jookia's Avatar
    July 2007
    6,614 Posts
    Could somebody port this to Python or C++?
    Reply With Quote Edit / Delete Reply Australia Show Events Optimistic Optimistic x 1 (list)

  5. Post #45
    Venice Queen's Avatar
    January 2011
    187 Posts
    Could somebody port this to Python or C++?
    You could contribute bindings to the project

  6. Post #46
    Gold Member
    Jookia's Avatar
    July 2007
    6,614 Posts
    How do you bind something in C# to Python or C++?

  7. Post #47
    deloc's Avatar
    January 2008
    1,125 Posts
    How do you bind something in C# to Python or C++?
    ironpython, embedded mono, or porting.

  8. Post #48
    DeadKiller987's Avatar
    September 2010
    1,092 Posts
    Didn't steamCooker release some application that could be used t o talk to friends when having only an empty steam account. Then Valve fixed it. I thing it was on the Developer section on rin.
    Reply With Quote Edit / Delete Reply Lithuania Show Events Agree Agree x 4Funny Funny x 1 (list)

  9. Post #49
    Gold Member
    aVoN's Avatar
    December 2005
    2,754 Posts
    Could somebody port this to Python or C++?
    Yeah, and maybe Java so we finally can have a SteamFriends client for Android. But I think that will take too much time. It might be easier to use the knowledge of this lib and reimplement the necessary stuff for SteamFriends networking in Java.

  10. Post #50
    Gold Member
    DrTaxi's Avatar
    September 2009
    5,605 Posts
    http://opensteamworks.org/SteamKit/
    Sounds like it's supposed to be on Java at some point...
    Reply With Quote Edit / Delete Reply Germany Show Events Informative Informative x 1 (list)

  11. Post #51
    deloc's Avatar
    January 2008
    1,125 Posts
    http://opensteamworks.org/SteamKit/
    Sounds like it's supposed to be on Java at some point...
    it was on java at one point. that page is ages old.

    Edited:

    we consider that to be the dark ages of steamre.

    Edited:

    this is the renaissance.
    Reply With Quote Edit / Delete Reply Canada Show Events Funny Funny x 6Informative Informative x 3 (list)

  12. Post #52
    q3k
    Gold Member
    q3k's Avatar
    October 2009
    923 Posts
    Awesome job, although it begs for a C port... I guess I'll add this to my ToDo list.
    Reply With Quote Edit / Delete Reply Poland Show Events Agree Agree x 5 (list)

  13. Post #53
    deloc's Avatar
    January 2008
    1,125 Posts
    Awesome job, although it begs for a C port... I guess I'll add this to my ToDo list.
    or a c++ port unless you want to write/find a protobuf implementation in c.

  14. Post #54
    q3k
    Gold Member
    q3k's Avatar
    October 2009
    923 Posts
    or a c++ port unless you want to write/find a protobuf implementation in c.
    http://code.google.com/p/protobuf-c/ ..?
    Reply With Quote Edit / Delete Reply Poland Show Events Zing Zing x 4 (list)

  15. Post #55
    2014 SH Pun Award Nominee
    Dennab
    January 2009
    24,631 Posts
    Somebody write an Android port and I am going to sell my liver to get an Android device.
    Reply With Quote Edit / Delete Reply Czech Republic Show Events Agree Agree x 2Informative Informative x 1Optimistic Optimistic x 1 (list)

  16. Post #56
    Reply With Quote Edit / Delete Reply Canada Show Events Disagree Disagree x 2Optimistic Optimistic x 1 (list)

  17. Post #57
    Hates php
    high's Avatar
    May 2006
    2,377 Posts
    Voided, are you guys looking for help with the project?
    Reply With Quote Edit / Delete Reply United States Show Events Agree Agree x 1Friendly Friendly x 1 (list)

  18. Post #58
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    Help would be pretty awesome.

    12:55:23°pm [Nick] High6 is now known as High6|Driving

    Shouldn't you be paying attention to the road?! You could die. What help would that be?!
    Reply With Quote Edit / Delete Reply United States Show Events Funny Funny x 1 (list)

  19. Post #59
    deloc's Avatar
    January 2008
    1,125 Posts
    12:55:23°pm [Nick] High6 is now known as High6|Driving

    Shouldn't you be paying attention to the road?! You could die. What help would that be?!
    shouldn't you be using vanilla mirc?
    Reply With Quote Edit / Delete Reply Canada Show Events Dumb Dumb x 3Funny Funny x 1 (list)

  20. Post #60
    [h2]AND MY ACKSH[/h2]
    LieutenantLeo's Avatar
    June 2009
    2,791 Posts
    So, where do I download it.
    Reply With Quote Edit / Delete Reply Canada Show Events Funny Funny x 11Optimistic Optimistic x 1 (list)

  21. Post #61
    Gold Member
    PelPix123's Avatar
    August 2006
    3,412 Posts
    Wrong Thread

  22. Post #62
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    Still unable to get Steam2 auth servers for me. It happens in both 0.4 and when I compile the svn version.
    Can you launch Vapor in -debug and see what errors it spits out?

    Edited:

    For those of you totally without context: crossposting from http://www.facepunch.com/threads/105...m-Steam-Client

  23. Post #63
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    In an effort to increase the amount of documentation in the SteamKit library, I'm fleshing out the wiki with various articles here and there.

    The newest documentation can be found here: http://tracker.limetech.org/projects...e/wiki/Library. This part of the wiki tries to describe the library and the design patterns it uses.

    If you're interested in a "get started quick" article, I wrote up some example usage code with guided documentation.

    http://tracker.limetech.org/projects...e/wiki/Example

  24. Post #64
    Gold Member
    nicatronTg's Avatar
    July 2009
    4,403 Posts
    Fun, C# is so deadly close to Java, porting won't be hard at all really, just meticulous. It'd be fun to see this running on Java, for things like GoogleTV(!), Android, etc.
    Reply With Quote Edit / Delete Reply United States Show Events Agree Agree x 2Disagree Disagree x 1 (list)

  25. Post #65
    Gold Member
    ZeekyHBomb's Avatar
    June 2006
    3,598 Posts
    I'm playing with logging into Steam3 and I have a question about SteanClient Callbacks.
    Right now a polling system is used, but one could easily make use of events and just invoking it instead of pushing them on a queue in PostCallback.
    If that seems bad due to multi-threading (considering the queue-access is protected by a lock) you could simply use a System.Threading.AutoResetEvent and Set it in PostCallback.

  26. Post #66
    yngndrw's Avatar
    March 2008
    865 Posts
    Oops missed this thread. Very nice work.

    I haven't forgotten about my Web-Based-Steam-Chat project, just haven't gotten around to it yet. This library would work well with ASP.Net for it.

    Does the Steam protocol still have support for those old Steam games there used to be - I wonder if they could make a come-back with custom clients ?

    Edited:

    http://tirania.org/blog/archive/2011/Jan-04.html
    Reply With Quote Edit / Delete Reply United Kingdom Show Events Agree Agree x 1 (list)

  27. Post #67
    Hates php
    high's Avatar
    May 2006
    2,377 Posts
    I'm playing with logging into Steam3 and I have a question about SteanClient Callbacks.
    Right now a polling system is used, but one could easily make use of events and just invoking it instead of pushing them on a queue in PostCallback.
    If that seems bad due to multi-threading (considering the queue-access is protected by a lock) you could simply use a System.Threading.AutoResetEvent and Set it in PostCallback.
    I just noticed that, will look into changing it around.

  28. Post #68
    Gold Member
    ZeekyHBomb's Avatar
    June 2006
    3,598 Posts
    I just noticed that, will look into changing it around.
    My proposal:
    Code:
    Index: SteamKit2/SteamKit2/Steam3/SteamClient/SteamClient.cs
    ===================================================================
    --- SteamKit2/SteamKit2/Steam3/SteamClient/SteamClient.cs	(revision 172)
    +++ SteamKit2/SteamKit2/Steam3/SteamClient/SteamClient.cs	(working copy)
    @@ -9,6 +9,7 @@
     using System.Collections.Generic;
     using System.Linq;
     using System.Text;
    +using System.Threading;
     
     namespace SteamKit2
     {
    @@ -22,8 +23,9 @@
     
             object callbackLock = new object();
             Queue<CallbackMsg> callbackQueue;
    +		ManualResetEvent callbackAvailable = new ManualResetEvent(false);
     
    -
    +		
             /// <summary>
             /// Initializes a new instance of the <see cref="SteamClient"/> class.
             /// </summary>
    @@ -83,6 +85,41 @@
             }
     
     
    +		/// <summary>
    +		/// Gets the next callback object in the queue, or if none is available blocks until one is. This function does not dequeue the callback, you must call FreeLastCallback after processing it.
    +		/// </summary>
    +		/// <returns>The next callback in the queue. Will never return null.</returns>
    +		public CallbackMsg WaitForCallback()
    +		{
    +			CallbackMsg msg;
    +			
    +			do
    +				callbackAvailable.WaitOne();
    +			while( ( msg = GetCallback() ) == null);
    +			
    +			return msg;
    +		}
    +		
    +		/// <summary>
    +		/// Gets the next callback object in the queue, or if none is available blocks until one is. This function does not dequeue the callback, you must call FreeLastCallback after processing it.
    +		/// </summary>
    +		/// <param name="timeout">A TimeSpan that represents the number of milliseconds to wait, or a TimeSpan that represents -1 milliseconds to wait indefinitely.</param>
    +		/// <returns>The next callback in the queue. Might return null.</returns>
    +		public CallbackMsg WaitForCallback( TimeSpan timeout )
    +		{
    +			return callbackAvailable.WaitOne( timeout ) ? GetCallback() : null;
    +		}
    +		
    +		/// <summary>
    +		/// Gets the next callback object in the queue, or if none is available blocks until one is. This function does not dequeue the callback, you must call FreeLastCallback after processing it.
    +		/// </summary>
    +		/// <param name="millisecondsTimeout">The number of milliseconds to wait, or Timeout.Infinite (-1) to wait indefinitely.</param>
    +		/// <returns>The next callback in the queue. Might return null.</returns>
    +		public CallbackMsg WaitForCallback(int millisecondsTimeout)
    +		{
    +			return callbackAvailable.WaitOne( millisecondsTimeout ) ? GetCallback() : null;
    +		}
    +		
             /// <summary>
             /// Gets the next callback object in the queue. This function does not dequeue the callback, you must call FreeLastCallback after processing it.
             /// </summary>
    @@ -106,7 +143,9 @@
             {
                 lock ( callbackLock )
                 {
    -                if ( callbackQueue.Count == 0 )
    +				if ( callbackQueue.Count == 1 )
    +					callbackAvailable.Reset();
    +                else if ( callbackQueue.Count == 0 )
                         return;
     
                     callbackQueue.Dequeue();
    @@ -125,6 +164,8 @@
                 {
                     callbackQueue.Enqueue( msg );
                 }
    +			
    +			callbackAvailable.Set();
             }
    One thing to think about is what should happen if we wait for another callback (right now it'll block indefinitely), but none comes in, though the user could take care of that by setting a timeout, provided via two overloads.
    If there is some kind of exit condition that verifies that no callback will ever come in (does the handler remove itself when the internet connection gets lost?) one could just use another ManualResetEvent and instead of callbackAvailable.WaitOne(); use WaitHandle.WaitAny( { callbackAvailable, lastHandlerRemoved } ); and return null if it returns 1.

    Another thing I've noticed are unnecessary references in the SteamKit2 project, which are System.Core, System.Data and System.XML.
    Further more did it compile fine for me when targeting .NET 2.0; .NET 1.1 didn't work because apparently the Visual Studio 2008 project-file is not compatible with that.
    I'm by no means a C# or .NET whiz, but wouldn't it be better to set the target as low as possible to make it run on out-dated systems as well? Are there any benefits when targeting higher versions if you don't use any newly introduced features?

    Edited:

    Oh, and yet another small question, since one will most likely call FreeLastCallback after GetCallback, would it not be sound to add a boolean-parameter to GetCallback that decides whether or not to automatically dequeue the callback?

  29. Post #69
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    The main issue with blocking for callbacks is the fact that monitoring for callbacks should be done in the UI thread. This way there are no invoke calls or other cross-thread calls being done.

    As for the references to System.Core, SteamKit2's lowest .NET target is 3.5. There are various places in the code were we make use of the features provided by that version.

    If there is some kind of exit condition that verifies that no callback will ever come in (does the handler remove itself when the internet connection gets lost?)
    As long as a SteamClient instance exists, callbacks can potentially be posted.

    Oh, and yet another small question, since one will most likely call FreeLastCallback after GetCallback, would it not be sound to add a boolean-parameter to GetCallback that decides whether or not to automatically dequeue the callback?
    Sounds good, I'll add that next chance I get.

    As for some insight into where this whole design comes from, I borrowed the idea of callback polling from Valve's actual steamclient API. I've always been fond of it, so I implemented a similar thing with SteamKit.

    Generally their system of polling callbacks is used for games inside the main game loop, but the same can still be applied for windows applications with the WndProc.

    The only downside to this design is in WinForm apps in that emulating a WndProc-like update loop isn't very pretty, as shown in the Vapor example: http://svn.limetech.org/repos/steamr...por/Program.cs

  30. Post #70
    Gold Member
    ZeekyHBomb's Avatar
    June 2006
    3,598 Posts
    The main issue with blocking for callbacks is the fact that monitoring for callbacks should be done in the UI thread. This way there are no invoke calls or other cross-thread calls being done.
    I think this should be in the hands of the developer using the library, not the one developing it.
    Both methods, polling or waiting, would be kept in tact.

    Instead of providing the WaitForCallback methods you could also just implement the ManualResetEvent and provide access to it as a WaitHandle (so you can't Set or Reset from the outside).
    The boilerplate code to wait for this handle and then get a callback is really minimal and the design quite dynamic, since you can also choose to block until other handles are signaled, block just a certain time or not at all (WaitOne(0)), though if this is the case one would probably just directly call GetCallback and check for null, although it might be a little cheaper since you don't have to lock the queue.

  31. Post #71
    Hates php
    high's Avatar
    May 2006
    2,377 Posts
    Is there a reason you used a callback list over an event? Any reason you don't have a dispatch thread inside SteamClient that handles the callbacks instead of the Update loop?

  32. Post #72
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    Is there a reason you used a callback list over an event? Any reason you don't have a dispatch thread inside SteamClient that handles the callbacks instead of the Update loop?
    Because user code can implement that, if that's the type of functionality they require.

    Edited:

    I am, however, taking Zeeky's advice and I am implementing the WaitForCallback functions.
    Reply With Quote Edit / Delete Reply United States Show Events Friendly Friendly x 1 (list)

  33. Post #73
    Hates php
    high's Avatar
    May 2006
    2,377 Posts
    Because user code can implement that, if that's the type of functionality they require.

    Edited:

    I am, however, taking Zeeky's advice and I am implementing the WaitForCallback functions.
    What other functionality would someone want 0,o. And if they do want something different couldn't they just create a new class that inherits SteamClient?

  34. Post #74
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    I figured I'd update this thread with some of the latest happenings, so here goes.

    Vapor is still being developed and has a few new features compared to it's first release.

    The major update I wanted to mention is that SteamKit2 now has the functionality to connect to content servers and download game depots. Along with this, I've committed a new proof of concept project called DepotDownloader. You can view it here: http://tracker.limetech.org/projects...epotDownloader

    This program essentially allows you to download game depots similar to how hldsupdatetool downloads the files required to deploy source engine servers.

    All in all it's just another step forward in giving developers more opportunities to interact with the Steam network.

    One thing, however, that I've been curious about is if anyone is working on a port of SteamKit2 to another language. Be it Java or C/C++, there's a lot of potential out there to move SteamKit2 to another language, and I know a lot of people would want to see a steam client for their mobile platform of choice, so I wonder if anyone has attempted this.

    It would be a pretty massive undertaking, since the networking being Steam is very complex and has a lot of intricate details, but I'm curious nonetheless.
    Reply With Quote Edit / Delete Reply United States Show Events Informative Informative x 4 (list)

  35. Post #75
    Gold Member
    s0beit's Avatar
    August 2010
    2,787 Posts
    Good stuff.

    I can convert it to C++, if you want, when i have some time. I'm on IRC.
    Reply With Quote Edit / Delete Reply United States Show Events Agree Agree x 1 (list)

  36. Post #76
    Gold Member
    Colton Rappe's Avatar
    August 2006
    996 Posts
    Good stuff.

    I can convert it to C++, if you want, when i have some time. I'm on IRC.

    I am all aboard with this idea, I could use the C++ for a learning curve for my C++ Lessons.

  37. Post #77
    Gold Member
    Mattk50's Avatar
    April 2010
    7,094 Posts
    i would love it if someone here wanted to make a steam trillian plug in

    Edited:

    and so would the few thousand people who want one
    Reply With Quote Edit / Delete Reply United States Show Events Agree Agree x 1 (list)

  38. Post #78

    August 2010
    25 Posts
    Awesome
    now the linux community will develop a steam client that can't play games.
    but thats enough for me!
    Reply With Quote Edit / Delete Reply Anonymous Proxy Show Events Dumb Dumb x 4Agree Agree x 1 (list)

  39. Post #79
    Gold Member
    VoiDeD's Avatar
    August 2005
    840 Posts
    Small SteamRE update, but the project's main codebase has moved over to bitbucket.

    You can now find it here: https://bitbucket.org/VoiDeD/steamre

    Hopefully this will make it easier for other developers to contribute changes, patches, etc.

  40. Post #80
    Gold Member
    Jookia's Avatar
    July 2007
    6,614 Posts
    Does this violate the Steam Subscriber Agreement?

    http://store.steampowered.com/subscriber_agreement/

    Section E, for example.
    Reply With Quote Edit / Delete Reply Linux Australia Show Events Informative Informative x 1Agree Agree x 1 (list)