Monday, October 7, 2019

Defcon CMIYC 2019 Write-up

Rough hardware estimate

HardwareCountHardwareCount
2080 TI2107025
1080 TI2210606
1080159702
1070 TI:7CPU Physical Core:892

Software used

  • John the ripper + MPI
  • Passcovery Suite
  • Hashcat
  • Bcval
  • Hashtopolis
This year we had 15 members compete, we welcome 0xln, golem445, and John_Smith to the team. Tremendous thank you to Korelogic for hosting the 8th Defcon Crack Me If You Can, it was a blast this year as it contained interactive elements involving a dynamic workflow. This is quite different from what we expected and was welcomed with open arms.
After seeing that the test hashes contained usernames which involved matching, we suspected that pairing up different hashes and passwords with similar usernames was going to be needed this year. Prior to the contest, we developed a matching utility that would handle this operation for us. The pair tool was used throughout the contest to generate scrypt Hash:potential-pass pairs for us to run using single-mode style attacks.
We here able to get quite high completion rates on the “easy” hashes.
What is Bcval
Our Bcrypt Validator (Bcval for short) was originally created for the Ashley Madison project Cynosure Prime worked on back in 2015. It is an optimized single-mode cracker designed to work with hash:[potential]password pair and supports rules and case twiddling. This was quickly adapted to support scrypt during the contest.
While our team initially used john to attack the scrypt hashes, once Bcval was available we switched across as we noticed it was quicker in the workloads we benchmarked. The performance increase highlighted with the vertical red line can be seen in the contest performance graphs.
We are unable to directly produce a Bcval vs John comparison as we are running into a strange issue when using --single=none. It appears john is still using some sort of rules or is testing extra candidates (note this could be due to a PEBCAK error). On a Ryzen 2700x Bcval was running at about 240 h/s. However when running two identical hashlists john appears to run for quite some time even though it was reporting 220+ h/s.
It is unclear why we see such a discrepancy between the performance of the two tools as they are using the same hashing code. We believe this is due to the way we handle the hashes, this includes the ability to terminate jobs when a cracked hash is found as well as using a different threading model. Bcval does not use OpenMP nor is it using the ROM optimizations for the scrypt algorithm. As an afterthought to the contest, a rule candidate cache would have added nice gains for certain rules. The rule cache would effectively temporarily store all generated rule candidates for that single plain and check whether it has been tested, skipping the hash routine where applicable. Considering the slow speed of the hashing algorithm, the processing cost to store candidates and performing lookups would be negligible but would provide huge benefits.
As we had a pairing tool, this was used to crack the password changes. Pair-ception was carried out with the cracked passes along with the accounts affected. The cracked user:scrypt:plains were paired with user:[new]scrypt to provide user:[new]scrypt:[potential]plain which was then cracked by using simple rulesets such as best64 in single mode with Bcval.
While we did notice the avatar, WOLF, core and other hashsets we were not able to properly figure out the patterns for them. It appeared we were trying more complicated things than needed.
Early on, we ran into an issue where the system did not like empty hashes (we found several hashes that had blank plains), this was quickly rectified by korelogic. Due to focussing on cracking the hashes we decided not to recode our system to re-validate hashes on our end prior to submitting them and did run into a couple of submission mistakes where we accidentally submitted bad hashes.
Our team generally prefers the whatever approach, as any member can work on whatever hashlist or task they want. However, for this particular competition due to the added layer of hash pairing and password changes, some team co-ordination would have helped, especially when we had members who were only able to join in mid-way.
It was interesting to see the use of single mode password attacks and we were lucky to be familiar with this approach. It was very refreshing to see a dynamic element added to a traditional hashing cracking contest with the pairing and password changes.
Congratulations to Team Hashcat for bringing their A-Game and a massive thanks to Korelogic for organising Crack me if you can 2019.

Tuesday, May 21, 2019

CracktheCon Year 1

Over the years the members of the group have participated in quite a few password contests. Positive hack days’s Hashrunner, Defcon | Derbycon’s Crack Me If You Can, and SaintCon’s Pcrack. In fact, Hashrunner was the first contest Cynosure Prime competed in as a team. We love password cracking, especially come contest time. There is something special about the team atmosphere during contests, the problem solving, the ideas that come from those problems, the group chat, and light-hearted jokes and conversation between discoveries. It’s fun enough said.

The group decided it would be pretty awesome to design a password cracking competition of our own. Our member winxp5421 has been attending and speaking at Cyphercon for the last couple of years and seemed like a great fit for what we wanted to do. So discussions started with the con owner Mike Goetzman about putting together a password cracking village. Mike is an enabler of the best kind. You tell him of something you would like to do and the man will push you and help any way he can to make that a reality. We can't thank him enough for his support! Thank you, Mike!

A lot of thought was put into designing the challenges so that the street teams could participate with little experience and or little hardware, while still keeping things hard enough so that more experienced teams wouldn’t just blitz through the challenges. There is a difficult balance to achieve with this as challenges cannot be so difficult that no one solves them. The pro team challenges contained significantly tougher hashes and for certain challenges required teams to think outside the box such as using techniques to gather hashes or cracking hashes with no mainstream cracker support.


We used Google Cloud Platform for our server hosting. This decision was mainly because of the free $300 Google Cloud Credit you get as a trial. It took us just under a year to design, develop, test, and run the contest (In our free time of course). That $300 credit was just enough for us to complete the challenge without paying for anything out of our pockets for computing resources ( Thanks Google!). We don't have sponsors of any kind so everything had to be done on the cheap. This was a great way to help kick off the contest with no upfront cost. Come contest time we had 2 servers one for the DNS challenge (more on that later) and the other was running the main contest site. The main contest site had 8 CPU cores, 16GB of ram, a 128GB SSD, and a 500GB HDD. This proved to be WAY overpowered during the contest even with team submissions containing lots of duplicate “founds”. This was mainly due to the “pre-filter” designed by hops to catch incorrect and duplicate submissions before team submissions hit the Database. This pre-filter saved us on performance issues big time.

We found out the theme of Cyphercon this year was “Hidden in plain sight”. Hmm, hiding hashes, founds, etc. in plain sight… not the easiest but, I think we did well-designing challenges to conform to the theme as much as we could.

The street challenges ranged from easy to rather difficult. Catchall, oui:2C3033 and Vision being the easier lists.
YourNameRocks, YouLikeToSing? Being the intermediate difficulty lists
Brainkiller and phytology being the most difficult lists given to the street teams.

Vision was one of the easiest to pick up on and crack. This list was md5 and plains consisted of TV show names. Pulled from the tvdb list.

Catchall consisted of sha1, md5 as the hash algorithms and used names of Pokemon in two languages as the base list and used some basic rules to modify the plains. The two algorithms with different hash lengths were used as a subliminal hint for the OUI:2C3033 hash list.

The Oui:2C3033 file name was, of course, the biggest hint of all for the OUI hashes. A quick google search would reveal that this list had something to do with Netgear routers. An even further google search would reveal that by default Netgear routers have a well-known wifi password scheme of “adjective noun 1-3 digits”. However, we can't just make things that easy now, could we? CsP specializes in the strange and obscure when it comes to database leaks. One of the things you quickly discover when dealing hashes in the wild is just because a hash looks like one algorithm it may not be or even worse it might be two different algorithms that a look exactly the same. The OUI list was actually two hash types that looked exactly the same sha1 which most teams picked up on very quickly but, the less discovered algo was mysql5. This challenge was about teaching our street teams that just because a hash looks like one thing does not mean it is. I think that lesson is something all of our hash cracking community should operate under.

YourNameRocks consisted of md5apr hashes. The plains were pulled from the very well known RockYou list as well as a facebook names list. The baselists were also modified with some basic rules

YouLikeToSing? As the name suggests contains lyrics but, with some text scrambling and tougher rule manipulation.

Brainkiller took the same concepts as the pro team Brainkiller hashes with just easier cost factors and easier plains.

Phytology. Oh, how the Phytology list gave our street teams headaches. This list clearly gave everyone trouble. It's the only list with zero that is right zero teams finding even one single hash. That is more our fault than the players. It was just too tough for teams. This list was designed to be the “great equalizer”. Just in case we had a team that was not playing in the correct weight class and it was too much, a little over the top we could say. Like the name suggests this list is based on plant names but not layman plant names, no the scientific names for plants with ?d?d?d and ?d?d?s appended to them. It was just too much and we are sorry. Chalk it up as a design/ creation learning experience.

InTheZone:

 You know what would be a great algo to hide in plain sight? NSEC3. It's always in plain sight. The hint or start of this challenge was simply “dig +dnssec txt starthere.crackthecon.0x23.pw”. This challenge was all about walking a DNS zone and picking up NSEC hashes along the way. Cracking those NSEC hashes and submitting solutions.

Plzadd2hc:

This list was obtained by hitting the TXT records from InTheZone. Once you found a hash by querying the DNS server. The server would return unfortunate news, an Argon2 hash. Those that found them were unable to use hashcat to crack these. Seen as the nature of these hashes are rough going the list used the same plaintexts as the NSEC3 hashes with some case twiddles.

Brainkiller:

consisting of 4 levels:
- L1: easy plains which are crackable without much previous information. This list also had salts that where base64 encoded ASCII art as hints.
- L2: one static salt for all hashes, more difficult passwords used
- L3: all cost factors set to 19, but the hashes are generated with a cost in a lower range. Plains should be medium difficulty.
- L4: salts contain information which founds from L1-L3 are taken as basewords and concatenated with slight modifications.

BreakingBad:

 These hashes where Bcrypt cost 10. This list has a distinct science theme consisting of elements and words sampled from the alchemy game.

DESleppard:

 Lyrics from various artists including Justin Bieber with whacky mutations and reshuffles.

AprilFools:

 Words from various sources with rules that hashcat cannot handle mixed in.


Hiddeninplainsight:

 This challenge was really cool. We generated 7z archives where the passwords to the archive was hidden in the CRC values of the files inside of the archive itself. A literal Hidden in plain sight challenge. The first archive’s password was simply the crc32 of the file inside. While the other two were a little more devilish but, once you hit the final archive you would be given the FantasticFour hashes.

FantasticFour:


Generally, one easy hash is included to help crackers reduce the wordlist
- 5x Bcrypt hashes (based on cyphercon/Wisconsin
- 5x MD5 hashes based on dog breeds with varying round (max 1337) hint is provided in one of the hashes which are suffixed with 1337
- 5x GRUB2 PBKDF hashes based on colors
- 5x DCC2 hashes based on stars  (the hint is provided if you apply rot13 to the usernames, of the hashes)

100% salt-free 110% hassle:

Sampled from isp, soundhound, hashkiller & domains list
Various names from names + wordlist combos use rules hashcat does not support
Sampling from sound_cloud, names, isp, domains txt
Apply ultramap, 1 char to many char remapping eg w -> \/\/ and vv, H => |-|
Apply bizzare half reverse
Apply Suffix first char
Apply Prefix last char
Apply rules to memory + non memory
Use mixed char swaps
Use blockinsert rule


SuperList:

nouns(len3-5)+? combo with superheroes (mutated),  superheroes+superheros(mutated) + rules


A lot of the passwords were basewords sampled from rockyou, fbnames, tmto while some were collected such as the superlist which was taken from the superhero directory or breakingbad which contained elements but also had words sampled from the alchemy game mixed in.  The most interesting part was the rules that were applied to the words. Many of the rules either did not exist or were not conventional rules including single char replacements as opposed to the general swap all, one char to many char mappings rather than one to one. Some trickier ones included forward blockclone but from any position, as opposed to the standard start and end blockclone, semi reversing words. Applying rules to memorized word/portion and also applying rules to the non-memorized portion. The idea was for teams to use conventional rules to discover the baselist or perhaps spot some of the non-conventional rules then apply these to the baselists.

We were surprised that while teams were able to unlock the fantastic four hashes, they either were not able to find the pattern or notice the hints. Each set of hashes from the fantastic four challenge contained at least one easy hash which was supposed to give the crackers some idea on where to look. There was a hint for the DCC2 hashes where the usernames suggested something was going on. The 13 was present to suggest ROT13 was used, if this was applied to the letters of the usernames, it would have spelled ‘star’ which is what that list was based on. Each set of hashes was based on a theme and the MD5 set was made extra hard by using varying rounds of MD5. After cracking the ‘easy’ MD5 that should have prompted users which theme to focus on and the ‘1337’ was supposed to indicate the upper bound of the rounds. It is possible we didn’t allocate enough points to entice users to crack these or perhaps made them too difficult as we wanted to justify the number of points allocated to these.

With the first CracktheCon Contest behind us, I have to say things went really well.The contest site itself was rock solid however, the contest was not without its problems both player facing and behind the scenes.

The first day of Cyphercon beer was spilled on hop’s laptop. So hops spent most of the first day disassembling his laptop a few times trying to get his keyboard functional again. It was quite the sight watching him rinse his keyboard off in the bathroom sink. Luckily the expensive parts of the laptop were not damaged and a replacement keyboard arrived a couple of days later. He’s now getting used to the US lazout, darn…, layout.

Our member winxp5421 has quite the server infrastructure in his basement. A lot of CsP members use these machines for testing, development, etc. These machines where not being used for the actual contest site but they were being used as logging and visibility of what was happening on the contest servers. The last few months leading up to the contest Winxp has been working on plans to finish his basement it just so happened that the only time the framers could come put the basement walls up was day one of the convention. That proved to be unfortunate as the framers destroyed the coaxial feed line to his cable modem leaving us in the dark with no immediate remedy. Seen as Winxp was at the con a couple of hours drive away from home this proved to be a difficult fix but, a few phone calls, a buddy-pal-guy conversation later and a good friend came to fix the problem. (Thanks, Nate!)  We lost a couple of hours of logging data but, the contest remained unaffected and business as usual.

We sent out a reminder email the day before the contest. Each team was supposed to get a personalized email using the team name they signed up with. Unfortunately, everyone got an email addressing the name “false”... whoops, a variable screw up. On top of that Winxp was incapable of sending a single email that was typo-free though the entire contest. We were starting to think the man had a stroke.

It’s always DNS:
A couple of hours before contest’s start we noticed that the generated plaintext to the NSEC3 hashes was not unique. Instead of 33100 hashes, we ended up with only 32492. As everything was already set up and well tested we decided to not fix that issue. It wasn’t really a problem because the worst case was to have multiple TXT records (Argon2 hashes) for one domain.

“Problems” with DEScrypt:

On the first day, we received complains that some DEScrypt hashes couldn’t be submitted. All the “faulty” submissions had at least one high ASCII character in it and we used only characters in the printable ASCII range. There's a fun thing in DEScrypt because of the DES parity bit. If you add 0x80 to any character it will generate the same hash. For example mDPKlCDttlrdw:CynoSure and mDPKlCDttlrdw:$HEX[4379ee6f53757265] Teams were submitting “solutions” that did not match our solution.


Brainkiller problems:

#1: When inserting the challenges into the DB we used batch insert (50 at once). There were a few plains which seemed to have an invalid encoding (not valid UTF-8). The inserts that had these invalid encodings failed so 50 hashes were not inserted because of one bad one. This resulted in some ~2k hashes which were in the hashlist, but not in the db. The missing hashes were inserted into the DB. The 2x multiplier was disabled for the missing hashes and the invalid UTF-8 was posted as do not crack.

#2: For level 1 hashes, the plains for the hashes were giving ASCII art of a song when being in order. To give a hint, we also wanted to have a song text ASCII art encoded in the salts of the hashes. The issue with this was the base64 variant used was not the correct one, therefore using the normal bcrypt base64 decode did not reveal this hint.

During a previous CMIYC contest. Korlogic had a live points graph. A lot of people like to see live graphs during the contests so we figured it would be a good idea to do the same for ours. This one year in particular hashcat was randomly assigned a color for the graph. This color happened to be pink. Soon after the contest start team, hashcat’s color changed to something with a darker color pallet. The RNG gods blessed hashcat with great color but, that color was changed we thought that was a shame and heckled team hashcat about it. It was rather strange that team hashcat’s color was also pink for our contest. Was it the RNG gods? Or an “if hashcat: then pink”? I'll leave that for you to decide.

HackTheNation was killing it in the upload game. Pushing more lines to the API than any other team. In total, the street teams had 654k hashes to crack. HackTheNation pushed 73.4 Million lines over the 48 hour contest period. Submitting hashes more than once did not gain teams any extra points but damn if that was going to stop HackTheNation from trying. We did not supply teams with a way to chunk their uploads to conform to the 50,000 line limit set by our API. Our guess as to why they pushed so many lines would be their submission script was simply using “tail -n 50000” to keep within our API call limits. We would love to hear why this team supplied is with so many duplicate hashes.
These frequent duplicate uploads made us really glad we used a pre-filter before hitting the DB with a query on each upload or we would have been in serious performance trouble. Even with all of the duplicate entries, our contest server was WAY overpowered. Our Load Averages stayed below 1.0 for the duration of the contest on an 8 core machine.

We want to thank Mike Goetzmen for all the hard work and dedication he puts into the con each and every year as well as the rest of the Cyphercon staff. We would also like to thank the participating teams. Turn out for the first password cracking contest we put on was better than we had hoped for. We hope you guys had fun with it and will join us next year. We have some ideas for further team torture.

Contest Solutions are available here

In veneration,
CsP