DUST 514 Technology Lab

 
^ Back to top

Topic is locked indefinitely.

3 PagesPrevious page123Next page
 

[Unofficial] Static Data Export Updated for Echo

Jump to first DEV post
Author
Fatal Absolution
Bleeding Sun Conglomerate
#21 Posted: 2015.02.06 20:51
Turkevich wrote:


But how do you get the files off the PS3 so you can analyze them?

Magic Cool

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Fatal Absolution
Bleeding Sun Conglomerate
#22 Posted: 2015.02.13 09:16  |  Edited by: THUNDERGROOVE
Just finished writing a tool to remove all the manual bits out of creating these dumps. Including the time of imaging my hard drive, decrypting it and generating the SDE file it's roughly 25 minutes now. Big smile

P.S. Adding an updated version for the latest sync today in the OP
P.S.S Looks like CCP took the loot table out of the strongbox type in Catma Cry

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Fatal Absolution
Bleeding Sun Conglomerate
#23 Posted: 2015.02.14 07:08  |  Edited by: THUNDERGROOVE
I accidentaled the drive I store my projects on.

No need to fear, rewrote all of my tools in just two days Lol and they are much better!

The tool I created pulls the backup, decrypts it using console keys pulls the proper game files into a directory.

Some of the files are encrypted in a Sony devised container, which was one hell of a pain in the ass to figure out what to do with P

Before, I had to manually patch the decompiled python code, move that all into a folder and add it to my PYTHONPATH. Now I have a script that overloads all the class methods which cause problems, mainly localization because I don't really want to **** around with the module they use for process management.

Was able to create SDE export by just importing my patch module and just importing their script, creating the object and calling the methods \o/

I don't plan on releasing this tool as I fear there could be some repercussions, however I don't mind releasing the SDE for CCP.

In the meantime, I'll finish my patch to the exporting object to pull all of the juicy stuffPirate

e: I also have the ability to generate SDEs in multiple languages as well if anyone has interest.
ee: Maybe not. Due to CCP using their own implementation of stackless Python they have methods internally that don't exist in official versions. I can patch around it, but there's so much code to wade through.

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Fatal Absolution
Bleeding Sun Conglomerate
#24 Posted: 2015.03.12 11:32
Pulling data. Get ready for a hotfix echo SDE ladies and gentlemen.

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Praetoriani Classiarii Templares
#25 Posted: 2015.03.12 12:38
ur good.

Amarr Victor

Join "PIE Ground Control" for secure Amarr FW syncing and orbital support

Kirjuun Heiian
#26 Posted: 2015.03.13 13:35
THUNDERGROOVE wrote:
Pulling data. Get ready for a hotfix echo SDE ladies and gentlemen.


How do you view these files?

Hnolai ki tuul, ti sei oni a tiu. Kirjuun Heiian.

I have a few alts.

Fatal Absolution
Bleeding Sun Conglomerate
#27 Posted: 2015.03.14 04:34
MINA Longstrike wrote:
THUNDERGROOVE wrote:
Pulling data. Get ready for a hotfix echo SDE ladies and gentlemen.


How do you view these files?

If you have your console keys you can decrypt harddrive backups P

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Nos Nothi
#28 Posted: 2015.03.17 13:54
Question: do you need all the synchronisations each time there is a new one?

Trying to explain.......

If there are three client synchs on days 1, 2 and 3, and you load up DUST each day, but miss your backup on day 2.....do you miss data?

If that doesn't make sense then I haven't understood how you're doing it so ignore me ;-)

There are 10 kinds of people in the world: those who understand binary, and those who don't

Fatal Absolution
Bleeding Sun Conglomerate
#29 Posted: 2015.03.17 23:08
IAmDuncanIdaho II wrote:
Question: do you need all the synchronisations each time there is a new one?

Trying to explain.......

If there are three client synchs on days 1, 2 and 3, and you load up DUST each day, but miss your backup on day 2.....do you miss data?

If that doesn't make sense then I haven't understood how you're doing it so ignore me ;-)

No because there will only ever be one synchronization per downtime

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Fatal Absolution
Bleeding Sun Conglomerate
#30 Posted: 2015.03.18 07:55
IAmDuncanIdaho II wrote:

If there are three client synchs on days 1, 2 and 3, and you load up DUST each day, but miss your backup on day 2.....do you miss data?

Completely missed your actual question. I would miss out on the opportunity to see the difference between two sets but I don't think it would really be a big deal.

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Nos Nothi
#31 Posted: 2015.03.18 13:35
THUNDERGROOVE wrote:
IAmDuncanIdaho II wrote:

If there are three client synchs on days 1, 2 and 3, and you load up DUST each day, but miss your backup on day 2.....do you miss data?

Completely missed your actual question. I would miss out on the opportunity to see the difference between two sets but I don't think it would really be a big deal.


Cool thanks, might have me a play around with some of this. Hopefully it's quite quick with a hybrid mechanical / SSD drive. I can see there may be some other useful applications with this too, depending on what's available.

There are 10 kinds of people in the world: those who understand binary, and those who don't

Brutor Vanguard
#32 Posted: 2015.03.26 18:21  |  Edited by: Greiv Rabbah
THUNDERGROOVE wrote:
CCP FoxFour wrote:
OK, you have me intrigued... how did you generate it?

The same way you guys do.
I had to modify some of the the catma code to get around the localization service which I was unable to get to work. Something about stacklesslib being a weird version. Nonetheless, people want it and you guys have been busy with other stuff.

e: I must say, the way you guys deploy "synchronizations" is really cool

e2: For clarification, a heavily modified version of catma.catmaDataExport


out of curiosity, and dont hate me for asking the obvious question, but you're using stackless and not vanilla python right?

edit: nvm. reading the rest of the thread made that question sound incredibly stupid

Sebiestor scout, MTAC pilot, Merc w/ a face

Fatal Absolution
Bleeding Sun Conglomerate
#33 Posted: 2015.03.28 06:12
Greiv Rabbah wrote:


out of curiosity, and dont hate me for asking the obvious question, but you're using stackless and not vanilla python right?

edit: nvm. reading the rest of the thread made that question sound incredibly stupid

Yeap! Luckly, I don't believe CCP uses their special snowflake version of Stackless for DUST so things mostly work.


Currently working on a tool to update my exports using mDescription and mShortDescription of official varients because I still don't understand CCP's broken ass localization system. Lol

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

The Hundred Acre Hood
#34 Posted: 2015.03.29 12:22
THUNDER!

... Another language? How's Java feeling?
..... I want to be a master too *le cry
Fatal Absolution
Bleeding Sun Conglomerate
#35 Posted: 2015.03.29 19:56
Shamarskii Simon wrote:
THUNDER!

... Another language? How's Java feeling?
..... I want to be a master too *le cry

Java? Nope P

The tool is written in Go. There are a few tools that do the decryption/SDAT dumping in C it uses.

Oh and a bunch of "patching" scripts in Python to make CCP's frameworks not **** the bed.

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Eden Claims Corp
#36 Posted: 2015.03.31 13:16
THUNDERGROOVE wrote:
There was a synchronization during last downime. Pulling files and generating a new SDE as I post this. I'll update the OP with a link.

My guess, 'Quafe' commandos?


Alright !!! Quafe commando's

Scouts and Swarms

New Edens Pawn Shop

Nos Nothi
#37 Posted: 2015.04.06 23:29
Hey THUNDERGROOVE

Quick Q if that's ok.

I got to the point where I could read the unencrypted archive files. If I don't have my console id and I'm on OFW 4.70....I don't think I can get at the encrypted files, right? Until something new comes along...I am assuming you got yours way back when it was possible with those tools.

There are 10 kinds of people in the world: those who understand binary, and those who don't

Fatal Absolution
Bleeding Sun Conglomerate
#38 Posted: 2015.04.07 21:10  |  Edited by: THUNDERGROOVE
IAmDuncanIdaho II wrote:
Hey THUNDERGROOVE

Quick Q if that's ok.

I got to the point where I could read the unencrypted archive files. If I don't have my console id and I'm on OFW 4.70....I don't think I can get at the encrypted files, right? Until something new comes along...I am assuming you got yours way back when it was possible with those tools.

I actually got mine with a teensy ++ 2.0 USB dev board.

There is a software proxy around that should work. Not quite sure of the name though.

You may have a tough time finding it. Sony likes to go after websites that host these files P

e: Did some searching. Looks like you can't get your keys using the software method anymore.
ee: The offset the keys are stored on your console is 0x80870 it's 16 bytes.

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

Nos Nothi
#39 Posted: 2015.04.08 07:23
THUNDERGROOVE wrote:
IAmDuncanIdaho II wrote:
Hey THUNDERGROOVE

Quick Q if that's ok.

I got to the point where I could read the unencrypted archive files. If I don't have my console id and I'm on OFW 4.70....I don't think I can get at the encrypted files, right? Until something new comes along...I am assuming you got yours way back when it was possible with those tools.

I actually got mine with a teensy ++ 2.0 USB dev board.

There is a software proxy around that should work. Not quite sure of the name though.

You may have a tough time finding it. Sony likes to go after websites that host these files P

e: Did some searching. Looks like you can't get your keys using the software method anymore.



Wow you *are* good!

Thank you for confirming my suspicion. I guess I could look at proxying via my PC and back-dating to OFW 4.66 - but I don't think I care that much for what I am trying to do, which is simply read the leaderboard data, which may not even be stored on the drive.

There are 10 kinds of people in the world: those who understand binary, and those who don't

Fatal Absolution
Bleeding Sun Conglomerate
#40 Posted: 2015.04.08 20:45  |  Edited by: THUNDERGROOVE
IAmDuncanIdaho II wrote:



Wow you *are* good!

Thank you for confirming my suspicion. I guess I could look at proxying via my PC and back-dating to OFW 4.66 - but I don't think I care that much for what I am trying to do, which is simply read the leaderboard data, which may not even be stored on the drive.

Reading leaderboard data is (probably) possible by editng the zips within the SDAT files with scripts to dump user tokens to disk at startup but I haven't tested it. You could use the keys with a tool they ship with the client for whatever reason called crest-knife(IIRC).

It's all really not worth it. CCP has very good security teams and they will quickly notice the odd requests, flag your IP, see what accounts are used by that IP, ban them all(including your EVE accounts)

e: If you have any more specific questions feel free to hit me up at markusthefish at the google mails P

The enemies of God stand broken before us. The light of the Reclaiming shines over them!

12/13/14 Never forget

3 PagesPrevious page123Next page
Forum Jump