1. Post #41
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    One can never truly be doing nothing, the word is merely used for explanations of things that are doing so little of something noticeable that it is negligible whatever else they are actually doing.

    Take for instance a person standing and "doing nothing" on a street corner. For one, we just noted that they were standing, that's something, they are also breathing, seeing, thinking, etc.
    I meant computers.
    A thread can do nothing (sleep).

    This library is awesome. I think I might try to make a new datastream using it instead of glon to see the benchmarks there as well.
    Heh, with the net library in GMod 13, datastream is no longer required.
    But yeah, I'm curious how well would it perform.

    Doing this is way out of the scope of what vON does. Leave this to tools like advanced dupe.
    I didn't say I'm going to do it in the vON library, did I?

    Look up bitshifting and string.format. These should get you starting with compressing numbers.
    Yeah... I really have no clue how to use those to compress numbers.

  2. Post #42
    TGiFallen's Avatar
    January 2010
    1,440 Posts
    I didn't say I'm going to do it in the vON library, did I?
    No, but you said you might so I was telling you that it's a bad idea to.

    Yeah... I really have no clue how to use those to compress numbers.
    Simplest way for me to put bit shifting for saving info would be an example with a short (0-255)
    you can save a char in a file as a plain number, such as
    Code:
    100
    Or you can save it as a character in the ascii set like this
    Code:
    string.char(100) --> would come out as "d" since d is mapped to 100 on the ascii chart
    It gets more complicated with larger numbers and I suck at explaining it. this and this http://en.wikipedia.org/wiki/Bitwise_operation should help more. Keep in mind that the bit wise functions that come with gmod are made for signed numbers iirc.

  3. Post #43
    Pycские Идиот
    LauScript's Avatar
    June 2010
    2,529 Posts
    keeping keys in order would be nice

  4. Post #44
    Map in a box's Avatar
    July 2009
    7,200 Posts
    I've been working on a Lua serialization library, glad to see others are too!
    Reply With Quote Edit / Delete Reply Windows XP Professional x64 United States Show Events Friendly Friendly x 2 (list)

  5. Post #45
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    keeping keys in order would be nice
    Why? The numeric keys are in order. Why would you need the others?

    Well, it's actually up to you.
    The __next metamethod of your table will take care of that.

    I've updated the code with some speed improvements, and a completely new way of splitting the table.

  6. Post #46
    Gold Member
    Donkie's Avatar
    July 2009
    1,272 Posts
    I've seen string.split somewhere, I think it was Löve2D, must've been implemented by them then.

  7. Post #47
    Pycские Идиот
    LauScript's Avatar
    June 2010
    2,529 Posts
    Why? The numeric keys are in order. Why would you need the others?

    Well, it's actually up to you.
    The __next metamethod of your table will take care of that.

    I've updated the code with some speed improvements, and a completely new way of splitting the table.
    Oh i misunderstood, for some reason i thought you said that numeric keys didn't get put in order or something, sorry.
    Reply With Quote Edit / Delete Reply Windows 7 United States Show Events Friendly Friendly x 1 (list)

  8. Post #48
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    I'm looking for help. My current method of escaping strings is proving outrageously slow.
    Does anyone know a better way of escaping the strings?

    Code:
    	json_example = {
    		menu = {
    			header = "SVG Viewer",
    			items = {
    				{ id = "Open" },
    				{ id = "OpenNew", label = "Open New" },
    				nil,
    				{ id = "ZoomIn", label = "Zoom In" },
    				{ id = "ZoomOut", label = "Zoom Out" },
    				{ id = "OriginalView", label = "Original View" },
    				nil,
    				{ id = "Quality" },
    				{ id = "OpenNew" },
    				{ id = "Mute" },
    				nil,
    				{ id = "Find", label = "Find..." },
    				{ id = "FindAgain", label = "Find Again" },
    				{ id = "Copy" },
    				{ id = "CopyAgain", label = "Copy Again" },
    				{ id = "CopySVG", label = "Copy SVG" },
    				{ id = "ViewSVG", label = "View SVG" },
    				{ id = "ViewSource", label = "View Source" },
    				{ id = "SaveAs", label = "Save As" },
    				nil,
    				{ id = "Help" },
    				{ id = "About", label = "About Adobe CVG Viewer..." }
    			}
    		}
    	}
    Takes 1.9 seconds to complete 5000 cycles.

    Code:
    von_complex_example = {
    	    1, -1337, -99.99, 2, 3, 100, 101, 121, 143, 144, "ma\"ra", "are", "mere",
    	    {
    	        500,600,700,800,900,9001,
    	        TROLOLOLOLOLOOOO = 666,
    	        [true] = false,
    	        [false] = "lol?",
    	        pere = true,
    	        [1997] = "vasile",
    	        [{ [true] = false, [false] = true }] = { [true] = "true", ["false"] = false }
    	    },
    	    true, false, false, true, false, true, true, false, true,
    	    [1337] = 1338,
    	    mara = "are",
    	    mere = false,
    	    [true] = false,
    	    [{ [true] = false, [false] = true }] = { [true] = "true", ["false"] = false }
    	}
    Takes 1.3 seconds to complete 5000 cycles.

  9. Post #49
    KatNotDinner's Avatar
    June 2011
    792 Posts
    What are you talking about, string.split doesn't even exist.
    Then you say that the function is brought by datastream? Cause there's no other explanation for it being in the wiki.

    http://maurits.tv/data/garrysmod/wik...index9f52.html

  10. Post #50
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Then you say that the function is brought by datastream? Cause there's no other explanation for it being in the wiki.

    http://maurits.tv/data/garrysmod/wik...index9f52.html
    Why would that even matter?
    Nothing useful can come out of debating over this.
    Reply With Quote Edit / Delete Reply Windows 7 Romania Show Events Dumb Dumb x 1 (list)

  11. Post #51
    PENISCORP DIRECTOR
    Gran PC's Avatar
    August 2007
    3,110 Posts
    Then you say that the function is brought by datastream? Cause there's no other explanation for it being in the wiki.

    http://maurits.tv/data/garrysmod/wik...index9f52.html
    How about you go ingame and test if it exists?!

  12. Post #52
    KatNotDinner's Avatar
    June 2011
    792 Posts
    How about you calm down and think that I might've used this function in GMod 12 when datastream still came with gmod? I used to use the function and I found out that it came with datastream recently.

  13. Post #53
    taste the salty dong
    Remscar's Avatar
    September 2009
    1,822 Posts
    VON seems to fail when i run
    print(von.deserialize(von.serialize({1,2,3})))
    
    line 134: Reached end of string, incomplete number definition.
    Reply With Quote Edit / Delete Reply Windows 7 United States Show Events Funny Funny x 1 (list)

  14. Post #54
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    VON seems to fail when i run
    print(von.deserialize(von.serialize({1,2,3})))
    
    line 134: Reached end of string, incomplete number definition.
    Fixed. Thanks for reporting!
    Reply With Quote Edit / Delete Reply Windows 7 Romania Show Events Useful Useful x 1 (list)

  15. Post #55
    taste the salty dong
    Remscar's Avatar
    September 2009
    1,822 Posts
    Fixed. Thanks for reporting!
    Awesome on the fast fix, i was working on my gamemode and was using vON to store character data in a MySQL database.
    Reply With Quote Edit / Delete Reply Windows 7 United States Show Events Winner Winner x 1 (list)

  16. Post #56
    Gold Member
    Divran's Avatar
    April 2008
    2,624 Posts
    Found a bug. When using this to transfer data from the client to the server, all " are changed into ' (It seems console commands do it, don't know why)
    This makes von print this error:

    Code:
    [...\von.lua:81] vON: Malformed data... Can't find a proper type definition. Char#1:'
    And I don't think you have any code for escaping ', if that's necessary.

    EDIT: A temporary fix I will use until you fix this is gsub:
    datastr = datastr:gsub( "'", '"' )
    but I'd rather not have to do that.


    And what if the data I send is supposed to have both a ' and a ", and they have a different meaning?

  17. Post #57
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Found a bug. When using this to transfer data from the client to the server, all " are changed into ' (It seems console commands do it, don't know why)
    This makes von print this error:

    Code:
    [...\von.lua:81] vON: Malformed data... Can't find a proper type definition. Char#1:'
    And I don't think you have any code for escaping ', if that's necessary.
    's are not escaped and they're not used as type definitions. This is very weird.
    Can you show me the table you're serializing and the resulted vON code?

  18. Post #58
    Gold Member
    Divran's Avatar
    April 2008
    2,624 Posts
    's are not escaped and they're not used as type definitions. This is very weird.
    Can you show me the table you're serializing and the resulted vON code?
    Any table of strings does the same. The resulting vON code is fine. It only breaks when sending it through console commands.

    Code:
    {"test","hurr durr"}
    EDIT: I think I have a solution. I'll use E2Lib's E2Lib.encode. A function which was made for encoding chars that normally break when sending over console commands. This should fix all problems.
    EDIT 2: Yep that fixed it.

  19. Post #59
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Any table of strings does the same. The resulting vON code is fine. It only breaks when sending it through console commands.

    Code:
    {"test","hurr durr"}
    I'm sure the problem is not with vON.
    If you send vON code through a console command, won't you have to re-escape it?

    Edit: Oh, now I've seen your edit.

  20. Post #60
    Gold Member
    Divran's Avatar
    April 2008
    2,624 Posts
    Yeah it never was your fault :) Ah well, talking (edit: err typing?) out loud helped me figure it out, as usual.

  21. Post #61
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Yeah it never was your fault :) Ah well, talking (edit: err typing?) out loud helped me figure it out, as usual.
    Are you going to use vON in Wiremod?

  22. Post #62
    Gold Member
    Divran's Avatar
    April 2008
    2,624 Posts
    I'm looking for a replacement for glon, yes. When I added includes to E2, I needed to be able to send multiple files. The easiest way to do this is to encode it in glon, split up the glon string into umsg (or console command) sized chunks, and send them. But if your E2 has approximately 5000 lines of code, glon.decode causes the entire server to freeze for several seconds. I wanted to try your vON in the hope that it didn't have this problem. It does... I tried sending 10 000 lines of
    Code:
    #AAAAAAAAAAAAAASSSSSSSSSSSSSSDDDDDDDDDDDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    and after freezing gmod for over 30 seconds, it errored with
    Code:
    Expression 2 upload failed! Error message:
    not enough memory
    Help?

    EDIT: If you're traversing the entire string, looking for separators, you need to stop doing that and instead use string.find or match or similar. That could fix it.

  23. Post #63
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    I'm looking for a replacement for glon, yes. When I added includes to E2, I needed to be able to send multiple files. The easiest way to do this is to encode it in glon, split up the glon string into umsg (or console command) sized chunks, and send them. But if your E2 has approximately 5000 lines of code, glon.decode causes the entire server to freeze for several seconds. I wanted to try your vON in the hope that it didn't have this problem. It does... I tried sending 10 000 lines of
    Code:
    #AAAAAAAAAAAAAASSSSSSSSSSSSSSDDDDDDDDDDDAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
    and after freezing gmod for over 30 seconds, it errored with
    Code:
    Expression 2 upload failed! Error message:
    not enough memory
    Help?
    Ah damn it. Looping so much shit character by character is obviously going to be slow. :C
    For storing this much data, you need something else.

    If you wish, I'll start working on a filesystem or archive-like format for serialization of large datas..
    It will be slow-ish for small tables, but I promise good speeds for such large data.
    It's funny because I've actually planned to do it this afternoon.

    Edit: Wait, before that, I will try your idea.

  24. Post #64
    Gold Member
    Divran's Avatar
    April 2008
    2,624 Posts
    Ah damn it. Looping so much shit character by character is obviously going to be slow. :C
    For storing this much data, you need something else.

    If you wish, I'll start working on a filesystem or archive-like format for serialization of large datas..
    It will be slow-ish for small tables, but I promise good speeds for such large data.
    It's funny because I've actually planned to do it this afternoon.
    Couldn't you just edit vON to handle long strings as good as small ones?

    EDIT:
    Edit: Wait, before that, I will try your idea.
    ok

  25. Post #65
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Couldn't you just edit vON to handle long strings as good as small ones?

    EDIT:


    ok
    Well, I'll be damned. It's actually faster than the old method. The JSON example scored 0.15 seconds less.
    I've updated the file. I hope it works.

    I'll look into using the same method to parse the numbers. :D

    Also to note, it broke compatibility with old vON code.
    \ is no longer escaped.
    The nice serializer no longer exists. It's useless anyway.


    Edit: There goes the first Dumb rating.

  26. Post #66
    Gold Member
    Divran's Avatar
    April 2008
    2,624 Posts
    vON is now in wiremod. :)
    Reply With Quote Edit / Delete Reply Windows 7 Sweden Show Events Winner Winner x 8 (list)

  27. Post #67
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Old:
    Code:
    GLON:
    	5000 encoding/decoding successions took 4.3902492523193 seconds to finish.
    	Length of final (probably overmutilated) data: 597.
    
    JSON:
    	5000 encoding/decoding successions took 2.1211223602295 seconds to finish.
    	Length of the final (100% mutilated) data: 545.
    
    vON:
    	5000 encoding/decoding successions took 2.0071144104004 seconds to finish.
    	Length of the final (100% healthy) data: 583.
    [Finished in 8.6s]
    New:
    Code:
    GLON:
    	5000 encoding/decoding successions took 4.2862453460693 seconds to finish.
    	Length of final (probably overmutilated) data: 597.
    
    JSON:
    	5000 encoding/decoding successions took 2.1321220397949 seconds to finish.
    	Length of the final (100% mutilated) data: 545.
    
    vON:
    	5000 encoding/decoding successions took 1.4760837554932 seconds to finish.
    	Length of the final (100% healthy) data: 583.
    [Finished in 7.9s]
    25% faster thanks to Divran.
    Reply With Quote Edit / Delete Reply Windows 7 Romania Show Events Winner Winner x 6 (list)

  28. Post #68
    likes men
    Python1320's Avatar
    May 2007
    1,717 Posts
    --snip--
    Reply With Quote Edit / Delete Reply Windows 7 Finland Show Events Agree Agree x 1 (list)

  29. Post #69
    hydral1k's Avatar
    June 2011
    71 Posts
    Very neat. Would be really nice if you added support for other Gmod datatypes. :)

    Edit:
    Nice find.

  30. Post #70
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Very neat. Would be really nice if you added support for other Gmod datatypes. :)
    It's on my to-do list.

  31. Post #71
    T3hGamerDK's Avatar
    January 2011
    2,551 Posts
    I'm glad to see things such as these come up, instead of people constantly relying on older technologies as time passes by (I'm looking at you, X.Org).
    Hopefully, developers around will adapt this, or include their own and equally fast versions, instead of still relying on GLON or the like.
    Reply With Quote Edit / Delete Reply Windows 7 Denmark Show Events Agree Agree x 2 (list)

  32. Post #72
    metromod.net
    _Chewgum's Avatar
    April 2010
    2,216 Posts
    Got an error trying to serialize a table containing vectors:

    Code:
    ] lua_run PrintTable(dusk.zombieSpawnpoints)
    > PrintTable(dusk.zombieSpawnpoints)...
    1	=	7806.8506 7943.6479 992.0000
    2	=	7655.5376 8504.5400 992.8785
    3	=	7739.3364 8230.2705 992.0000
    4	=	8018.0410 7831.1504 992.0000
    ] lua_run dusk:SaveZombieSpawnpoints()
    > dusk:SaveZombieSpawnpoints()...
    [gamemodes\dusk\gamemode\modules\sv_von.lua:99] attempt to call field '?' (a nil value)

  33. Post #73
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Got an error trying to serialize a table containing vectors:

    Code:
    ] lua_run PrintTable(dusk.zombieSpawnpoints)
    > PrintTable(dusk.zombieSpawnpoints)...
    1	=	7806.8506 7943.6479 992.0000
    2	=	7655.5376 8504.5400 992.8785
    3	=	7739.3364 8230.2705 992.0000
    4	=	8018.0410 7831.1504 992.0000
    ] lua_run dusk:SaveZombieSpawnpoints()
    > dusk:SaveZombieSpawnpoints()...
    [gamemodes\dusk\gamemode\modules\sv_von.lua:99] attempt to call field '?' (a nil value)
    I didn't add support for GMod types yet.
    Imma do it now, tho. Don't know how long is it going to take.

    Edit:
    Here is the development version of vON for GMod.
    Please tell me the results.
    Reply With Quote Edit / Delete Reply Windows 7 Romania Show Events Useful Useful x 1Lua King Lua King x 1 (list)

  34. Post #74
    hydral1k's Avatar
    June 2011
    71 Posts
    Looking good. Is there any plans for Color obj?
    Reply With Quote Edit / Delete Reply Windows 7 United States Show Events Agree Agree x 1 (list)

  35. Post #75
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Looking good. Is there any plans for Color obj?
    As far as I know, Colors are just tables, so they should be safely serialized and deserialized.

    Edit:
    I need feedback on the GMod development version.
    Please tell me if there are any errors or more types you need in.

    Edit2:
    I have to go to sleep, so I'll read the feedback tomorrow (in aprox. 10 hours).

  36. Post #76
    metromod.net
    _Chewgum's Avatar
    April 2010
    2,216 Posts
    As far as I know, Colors are just tables, so they should be safely serialized and deserialized.

    Edit:
    I need feedback on the GMod development version.
    Please tell me if there are any errors or more types you need in.

    Edit2:
    I have to go to sleep, so I'll read the feedback tomorrow (in aprox. 10 hours).
    You got 2 Vector (one is supposed to Angle) in serialize.

  37. Post #77
    KatNotDinner's Avatar
    June 2011
    792 Posts
    As far as I know, Colors are just tables, so they should be safely serialized and deserialized.
    True. The Color() function is defined in the lua files and it just creates a table.

  38. Post #78
    Facepunch's #1 Huskybutt
    James xX's Avatar
    July 2011
    2,092 Posts
    I didn't add support for GMod types yet.
    Imma do it now, tho. Don't know how long is it going to take.

    Edit:
    Here is the development version of vON for GMod.
    Please tell me the results.
    --	3 numbers separated by a comma.
    	["Vector"] = function(data, mustInitiate, isNumeric, isKey, isLast)
    		if mustInitiate then
    			if isKey or isLast then
    				return "v"..data.x..","..data.y..","..data.z
    			else
    				return "v"..data.x..","..data.y..","..data.z..";"
    			end
    		end
    
    		if isKey or isLast then
    			return "v"..data.x..","..data.y..","..data.z
    		else
    			return "v"..data.x..","..data.y..","..data.z..";"
    		end
    	end,
    
    
    --	3 numbers separated by a comma.
    	["Vector"] = function(data, mustInitiate, isNumeric, isKey, isLast)
    		if mustInitiate then
    			if isKey or isLast then
    				return "a"..data.p..","..data.y..","..data.r
    			else
    				return "a"..data.p..","..data.y..","..data.r..";"
    			end
    		end
    
    		if isKey or isLast then
    			return "a"..data.p..","..data.y..","..data.r
    		else
    			return "a"..data.p..","..data.y..","..data.r..";"
    		end
    	end
    

    just browsing through and noticed you had 2 Vector keys. I think the second one is supposed to be an angle :3
    Reply With Quote Edit / Delete Reply Windows XP France Show Events Late Late x 1 (list)

  39. Post #79
    Always the vGuy.
    vercas's Avatar
    September 2011
    841 Posts
    Ow darn.
    Fixed it.

    So, uh, how well does it work with the GMod types?
    I need to know so I can declare it a stable version.

  40. Post #80
    metromod.net
    _Chewgum's Avatar
    April 2010
    2,216 Posts
    Ow darn.
    Fixed it.

    So, uh, how well does it work with the GMod types?
    I need to know so I can declare it a stable version.
    I tested the vector, and it works, so I guess angle should work too.
    Reply With Quote Edit / Delete Reply Windows 7 United States Show Events Dumb Dumb x 1 (list)