New NFL Sim Improvements for the 2024 Season

Jordan: All right. What is going on, everybody? We've got week three of the NFL season coming in hot here this week. And while we've had a lot of product and front end changes kind of at the forefront in the first couple of weeks of the season, the models team has been putting in a ton of work on our NFL Sims and projections over the course of the off season and into the first couple of weeks of the season.

So I wanted to bring Matt and Will on today to talk a little bit about some of the improvements we've made in that area. Um, what some of the things we're seeing so far to start the season look like. Um, we've got a lot to talk to talk about here today. So first off, Matt, Will, how are you guys doing? Good.

Excited to be here. Doing great. Awesome. Cool. Well, do you, either of you guys just want to kick it off here with just an overview of some of the big improvements that we've made for this year?

Matt: Yeah, I'll start it off. So, uh, yeah, we haven't talked about it too much, um, on discord or anything, but we ended up making pretty huge changes to the model this year.

Um, and I guess just to, like, kind of get right to it, um, the Biggest thing is that we are much, much more, um, impacted by, and I don't want to say reliant on, but, uh, we, we are very much, um, using Vegas sports book, um, odds and props as kind of a primary input into the model. So, I mean, I think in the past, um, You know, we have used like the, the sports book lines as sort of guidelines or, um, safety rails and like, Oh, let's not get too far past.

Let's like, try to get the team totals to, to get closer. Um, and like looked at props, but not really the kind of incorporated them into the model. And we kind of realized, you know, as, especially as sports betting has become so big, um, there obviously are edges in betting props and like prop lines are not.

This like these perfect, um, perfectly efficient markets, but they're kind of this like wisdom of the crowds, um, data, especially as we get closer and closer to game time. Um, prop lines are very, very good. They are not easy. It's not easy to beat props. And we were thinking, like, why are we trying to, like, spend all this, like, effort trying to, like, get, um, better than props when we have this data that's really high quality and really good that we can actually incorporate into the model rather than trying to like kind of tiptoe around it.

So we made the decision to really like lean in to, um, sportsbook lines, you know, not just player stat props, but, um, you know, the, the money line total spreads all the, you know, these alt totals and like all basically anything that. You can bet on we're going to try to incorporate into the sim not as a kind of taking those and backing out of them, but using them in.

Um, as essentially like smart inputs into the SIM and then kind of, uh, after the SIM runs using them as ways to kind of, um, guide the, the end results. So I would say the really the biggest impact, um, practically, uh, for users is, is, you know, we are using player props, um, as, you know, Kind of inputs to adjust the rates that go into the SIM.

And so we're directly trying to get close to props. And so as props change, um, and as, as they get posted, we're going to be rerunning SIMs to adjust for that. And, uh, I think, you know, Will and I were talking a few days ago, just we're really, really excited about how things look, I think, you know, Will is probably more apprehensive about the idea of having this like really like sportsbook um prop based sim just because it's like you know there's a bit of a pride thing of like we built this really good model that wasn't based on that and um now we're kind of like trying to incorporate it into it but like When you look at the projections that we've been been publishing, how this is working, and the fact that we're still able to run the same core, like, play by play, Sam, that we had before.

Nothing really about the. The, like, actual, um. Like, framework of the SIM has changed. We've made a bunch of optimizations and speed improvements and stuff like that, but, like, we are still running play by play Sims that account for the context of the game that have. All of the game scripts and, and, and all the same like values of like the correlations and the upsize that we did before, but now we're using all of this data that was there freely available, really, um, that is like sharp and wisdom of the crowds based.

Um, and we're using that as, as really a way to like, shape our sims. So. Yeah,

Jordan: yeah, I have a few questions on that right off the bat, but will anything you wanted to add on there? I think it's interesting that mentioned you might have been a little apprehensive here with this direction. Um, just kind of your thoughts on on this project.

Wil: Yeah. So, I mean, I think maybe apprehensive isn't the right word, but like, there's definitely a lot of pride in the old model. Like, that was my, that was my baby. Um, I worked, you know, did a lot of cool stuff for it, um, but it really just makes a lot of sense to incorporate this data just in terms of, like, Okay.

You know, it makes sure that, like, we're going to have, like, on Sunday, like, good, smart projections, uh, and, like, we don't have to worry about, like, oh, there's some weird, like, uh, we were getting weird data on, like, this guy. And so he's, you know, a bazillion yards, like, it's a lot safer, uh, which is a lot less stress.

And it's still, like, the most important thing to me, which is what Matt touched on is, like, we're still doing the context of the game stuff. So, like, we may be smarter now with, like, this, uh, quarterbacks completion percentage, but it's still changing. If they're, like, throwing a halfback screen versus, like, uh, you know, a 30 yard deep.

Pass that still changes the completion percentage in the same and we're still accounting for all of those factors that change. Play to play that, like. You can't get those from props, but we can know. Okay. Well, we know that it's going to end up being about, you know, 55 percent completion. And so it's, I think it's really good that we're incorporating that data and staying true to, like, how we do the play by play.

Sims.

Jordan: Yeah, no, that makes sense. And I think there's a couple objections that I tend to hear from people when you start talking about Vegas and, um, one of them and you mentioned, you know, Being safer. Well, um, I think people's reaction to that at, at first hearing, that might actually be negative sometimes, right?

Like we're playing tournaments with a hundred thousand people. Like, I don't want to do something safe, like wisdom of the crowd. And maybe I want to fade the crowd. Right. Can you, either you guys talk a little bit about how, like, we're still kind of rate retaining an edge there, or how we still feel comfortable in building lineups that are poised to beat the hundreds of thousands of people you need to beat in a DFS contest.

What kind of relying on this data that we think. Is like safer or, or kind of more in line with the, the wisdom of the crowd.

Matt: Yeah. I mean, I think the first thing is that there's still a lot of, especially for NFL, there's a lot of people playing really bad lineups that are like, not either, not using projections, using bad projections, just like hand built based on players they like, you know, when you're going into the, especially like the big tournaments, um, like really probably the edge is just like you beat the bad.

Um, even if you're not the best player in the world, you can certainly be profitable in an NFL, just like playing good plays, even if they're safe. Um, but then there's also, of course, like the game theory aspect and why contest Sims are so valuable is that even if you're playing projections that are essentially the same as what the field is playing, you can beat the field based on game theory, based on lineup construction.

Um, Uh, you know, lineups that are correlated to have high upside and have players that have high upside. Uh, all that stuff is still valuable, even if like the mean projections are. Are safe and, and we're not like taking huge stands. Um, and then the last thing I think is that like, it's, I think this makes it actually a lot easier to take stands because you can, if you think that a prop is worth betting, then that means you think that the player projection is worth changing, right?

And so like. You can, I think before it was like, if you're using Sabers and we have a big stand on a player that's different than a prop, you're kind of forced into that play, like, you kind of are not, you're not forced. You can obviously change the projection, but, you know, we're going to like, say, hey, you need to be overexposed to this player.

And I think now, um, You know, we're more leaving it in the user's hands in terms of making those stands that are, that are, um, different than, like, what the markets think. Um, and we're saying here are, like, the sins based on, like, wisdom of the crowds. If you want to be different in the crowd, if you think that the prop line is wrong, if you think the crowd is wrong, whatever, you can change it.

Um, but, like. You know, we think that even if we can be slightly more accurate than props, there's higher risk. There's higher risk of like, oh yeah, we're a couple, you know, points DFS points off from what the prop implied projection would be. And now again, you're getting forced this like 50 percent exposure to a player that is, um, just high risk if we, if we are wrong, you know, and so, um, It's, I don't want to just focus on playing it safe because I think it's more than being safe.

I think that we're leaning into the, the wisdom of the crowds philosophy of like, you know, even if there are edges, like, in, you know, in any sort of prediction thing, like, very often. You take all of the data and all of these different opinions about players and like the prop sportsbook lines really are just like the aggregate of a bunch of different opinions about what players will do.

And, uh, that's kind of our philosophy is we're going to take that valuable data and, and use it. So.

Jordan: As we're looking at the different markets that we're pulling in, you mentioned like not only just the spreads and the money lines, but also alt totals, um, alt spreads, obviously the player props. How do we, um, assign a value to each of those different markets in terms of like how, how accurate their information is?

Obviously the liquidity of those different markets is going to be different. We probably have a different way of thinking about each of those. How does, how does that kind of come together?

Matt: Yeah. So we do prioritize the. Um, the, the, you know, more liquid markets that the money line total spread. Um, and so we're, you know, as we're, we're going to prioritize getting, like, the team totals.

Correct basically, 1st of all, um, and like, everything else is like, a little bit. Lower priority, but obviously for DFS, like the player. Stats do matter, you know, probably more than the team totals, but it kind of cascades down, right? If you get the team totals right, then it's, that's obviously going to affect the accuracy of all the players.

Um, you know, if we get the QB right, that'll affect all the accuracy of the wide receivers and, and, um, even the running backs too. So, um, You know, in general, like the, we have created a really, really cool approach technically to be able to like, take, you know, these hundreds and hundreds of different like bets and, um, kind of use them in a smart way.

Um, but yeah, we are basically focusing on like the, the stronger signals of the team, total game, total, uh, spread type. Markets 1st

Jordan: and I know it's only been 2 weeks, but, um, you mentioned at the start here that we've already seen some promising things. For the start of the season, what, what have we seen so far that that makes us excited about this here or looking at, like, projections or correlations or percentiles?

What are some of the indicators that were, um, we've made an improvement.

Wil: So, for me, the, the biggest 1 is with rookies. Um, yeah, those are just always a nightmare. Uh, like, the start of every season, it's just a nightmare to project them because it's, it's hard to deal, like, limited college data. Like, we have their draft position, but, like.

The market is like. Probably a bit bullish on some rookies and, you know, like, but in general, it's a way better baseline than what I think we were working with before. So, it's like, I know, unfortunately, none of the rookie QBs have thrown a touchdown, but I feel like our projections for them. The 1st, 2 weeks have been good because they've been in line with their props and what expectations are.

And I think that would have just been significantly harder with our old approach because of that lack of data. Whereas like now we're missing like the concrete. Data that the old model was built on, but we have so much data from the market, which is informed of their college performance of the system.

They're going into of coaching changes and all sorts of things that are a lot harder for us to get until they start showing up in data. That makes sense.

Jordan: Yeah, no, it absolutely does. What, what are some of the, again, I know, I know it's only been 2 weeks, but as the season goes along, what are some of the ways that, like, we quantitatively check this kind of stuff?

Like, what are some of the metrics that we're looking through over the course of the season to say, you know, okay, we got better in this way. By X amount

Wil: yeah, so, um, that's like a big thing that I focused on in the off season. 1 of my projects is building, uh, like a accuracy tracker, basically. Um, and so we, we do a couple of things, but the most important 1 to me is the distributions.

Um, so for each players, like, stat, like, all of their stats within the sim, including, like, fantasy points, but also, like, passing yards, rushes, um, rushing yards, et cetera, um, were, like, storing and keeping track of what our predicted percentiles was, was, yeah, for all of these stats, um, and then as well as the actual observed for that game.

And so this lets us go back and then compare, like, okay, how many time, like, what percent of the time was, like, the rushing yards under the 25th percentile that to be, like, right about 25 percent of the time. And so doing that, let's just like, check the calibration of all of the stats and everything like that.

On top of just, like, the standard things, like, R squared, or, you know, that kind of stuff for the means.

Jordan: Is there any reason to think that, like, are the distributions in particular would have gotten better with this project? Or is it more of the mean projections or just kind of everything just gets a little bit better across the board?

Wil: I think that it would, like, I'm not 100 percent sure on this take, like, Matt might disagree, but to me, I feel like the distributions probably got better. Just because of like the way that we're like handling the, like the total game distributions, like the old totals and everything like that. Um, I feel like that probably has like a rundown effect on making like the tails a bit wider, um, than we might've been considering before for players.

And so I think like, that's my gut thinking is that it will improve that. Um, and then definitely with mean predictions, um, that should definitely improve.

Matt: Yeah, I agree. I think the, like the means have probably had, you know, This update had a bigger impact on those. Uh, but yeah, especially using all of those, like, you know, all team total all spread game total lines, um, to kind of dial in the Sims after they run, uh, is going to have to A decent effect on those distributions and, you know, we'll see better, you know, a lot of the data.

Unfortunately, we didn't have for last year. So, you know, back, we, we did back test quite a bit in terms of, like, some of the, the player stuff, but we didn't have all of the data to be able to back test, like, fully. So it'll be good to see, you know, probably in a week or 2, be able to kind of look at what we have and, and, you know, See where there still is room for improvement.

Um, let's see where we're at with those distributions and calibrations. Just like I test wise, like correlations I know last year or in the past, we had some, um, I don't know if it's like issues or just some of like the correlations, especially between like opposing defenses, kicker defense, sometimes looked a little bit weird or just didn't quite match what we would expect where like, you want to have a positive correlation between like kicker and defense and, and, you know, Negative between opposing defenses or something like that.

And a lot of those numbers look much better just on the eye test when we're looking at the correlations this year. And that wasn't something that we were trying to do, um, necessarily, but I think it's just a lot of the improvements we made overall to the model. It, uh, helped improve those naturally.

Jordan: Yeah.

Yeah. No, that makes a ton of sense. And I, I mean, anecdotally, the, the correlations like seem to be also a little bit more specific. Like I think we've always done a pretty good job of like rushing quarterback, maybe a little bit less, um, correlated to his pass catchers that, uh, a pure pocket passer. Um, and that, and again, that's something that I think we've done a decent job of before, but it just feels one more degree.

Dialed into me. And I know, I mean, that that's, that's like a very qualitative take. Um, but just looking through, I've been poking around like things. We, we seem to be less general about how we categorize particular players and more specific of like, no, that actually seems like a very fine tuned. And I think that's probably like some of those prop markets of like, you know, completions, uh, receptions, um, like I longest reception props.

Like there's all these different ways that you can take in information from the markets here and. Have a better sense of what their outcomes actually look like.

Matt: Yeah, I mean, I do think that we've all like, we have always been specific with like, stuff like that. But I think, like you said, we're more accurate at like, you know, Lamar's rush attempts and, and touchdowns and stuff.

And like, you know, even just the difference between Someone a quarterback that rushes a lot, but doesn't get a lot of touchdowns versus, you know, like Hertz, for example, you know, the push push stuff, like, you know, he's going to have a pretty high, like touchdown per rushing yard ratio because like they, and that's going to show up in the props and it might not have shown up as well, um, without having that data.

Um, I think the other thing I do want to. To bring up that sort of like a little bit like more meta or higher level but like because we don't have to focus as much on like fixing like player level projections and like rates and all that stuff because we can rely on the props for that we can like focus a lot more of our time on the logic of the sim and getting those distributions and correlations right.

And not get distracted by like, Hey, why is this rookie quarterback projected for five points more than he should be? Because like, we know we can kind of rely on that stuff and we can really focus on what the markets are missing. Um, not where they're wrong, but just where, like, we don't have, there aren't really great markets for a lot of, for, like, correlations and, um, I mean, you could maybe use, like, same game parlay stuff, but, like, really, I think that's where this play by play sim is so valuable.

It's not in projecting the, each player's, like, passing yards per, you know, per, per completion or whatever. It's, like, In in actually giving us an accurate range of outcomes, accurate correlations, upside correlations and so will and Eric can, like, just focus on that. Get our play by placing as accurate as it can be.

And not spend all this time, like, fixing or just improving those inputs.

Jordan: Yeah, no, that makes sense. I know along with this. Has come a little bit of a way that we've changed our sim scheduling on, on not only Sundays, I guess, but also just over the course of the week, as we are getting information, how those projections change, I guess, starting with Sunday.

The big change we made is, um, not running kind of that, like, final sim, 60 minutes before kickoff, and instead just running Sims as additional information is coming in. Can you just kind of, I guess, talk a little bit about that as well, in terms of why that was changed and in relation to the. Listening to Vegas more here.

Matt: Yeah, so I mean, obviously if we're using Vegas as input, like those lines are changing, um mm-hmm . Constantly. And like the closer we get to the start of the game, the more accurate they're going to be. Um, that's really just sort of, that's sort of like one of the fundamental like truths. I mean, maybe it's not true in 100 percent of cases, but for the most part, you know, the closer we are to game time, that more accurate those, uh, lines are going to be, um, maybe with the exception of like Superbowl or something where like a lot of the soft money comes in at the end or something.

But, um, yeah, so, so. Basically, as that data changes, we want to make sure that, you know, our projections are accounting for those changes. Um, and it just not updating at all in the past, in the last hour is going to be missing some of that data. Obviously we, we do have to kind of balance that with like, having a not confusing experience for users where they don't know when they should build and stuff.

And that's something that's going to be partly just sort of talking about it more, educating people on like, build early and like kind of look at what's changing but you don't have to change like rerun things after every sim um it kind of just depends and that is you know it's a tricky balance um you know as a dfs player it is it's hard to like know to like build and then see a new sim come in and have to decide what you're going to do so i definitely get that but just sort of like We want to put it in users hands and let them make the decision rather than us making the decision of like, oh, we're just not going to run a SIM because nothing changed.

Um, so that's something that's sort of like an active process. We, the first couple of weeks, I think we didn't. Um, too much. We ran some, I mean, especially with like Evan Ingram, I think obviously that news was pretty late. We reran, but I think as the season progresses this week and moving forward, we will be running more Sims late, depending on what changes, um, you know, I think a good example, which we didn't account for, um, on Monday.

Because it was so late, but I think it was Covey, right? Um, the wide receiver, like it was announced that he was like starting like 10 minutes before lock or something at that point, we didn't have time to run a SIM anyway. But like, if that had gotten announced, say 30 minutes before, and the props had updated where like his receptions, his receiving hours, anytime TD, whatever changed, like we likely would have run a new SIM because that's important information that it changes, you His projection and changes kind of like the, the whole game.

And we wanna account for that. So that's the kind of thing that we will be looking at. I think it's going to be, you know, a a, we're gonna adjust that process as a team ourselves of like figuring out when to run Sims in that last hour. But like where the thresholds are data changing, we obviously don't wanna just run Sims for like one yard.

Changes, but there's gonna be some thresholds where the lines change, you know, if there's a bunch of sharp money that comes in, you know, in that last hour, that's shifting a, um, you know, passing yards prop from 275 to 260. Or something like that. That's like a that's a decent change. Or if there's like they post props late where someone might not have a like rushing yards prop.

And then, um, it gets posted late and we want to make sure that we're accounting for for that new data. So, um, and I do 1 last thing is that we're going to be. More transparent about actual data too, where you'll be able to, this isn't there yet, but in the app, be able to click on a game and like, see actually like, okay, these are the props that were used.

These are the lines that were used. And so, you know, again, really trying to focus on the transparency and giving Sabres and users the information and letting them make the decision. So if you can click in and see like, okay, there isn't like this SIM didn't have a Rushing yards prop for this player. Um, you know, and in those cases we're kind of using like player prior, like we're using historical data and estimating and doing our best, but that.

If as a user, you can see where there's data missing, it makes it a lot easier to kind of make an informed change. Um, and people like, okay, I, I have an opinion about. What this player is going to do here. There's no prop for them. That's probably a higher value change. Then, um, changing a player where we do have all the profits, so you can kind of see where there might be more uncertainty.

And the projections by looking at that. Um, And, uh, yeah, so,

Jordan: well, did you have something that you were going to add to that? You look like you were about to say something.

Wil: No, I feel like Matt

Jordan: covered it all. Okay, cool. Yeah. I mean, so I guess. On a similar note here, I think the kind of rollout of projections over the course of the week is a little bit different as well.

Um, obviously the slate locks on Sunday and that that's when our focus is being the most accurate when people are on, on Sunday mornings building their lineups. But for people that maybe are building throughout the week or kind of getting a sense of the slate, starting to research, um, what does kind of like our, our, our, I guess schedule look like as we're starting to run some of the early Sims over the course of the week.

Wil: Yeah, so it definitely gets a bit tricky dealing with the early week stuff. Yeah, when there's like, not props posted or not a lot of props posted and that's definitely something that we're still like, working on and tweaking and figuring out because it's just, you know, it's right now we're, we're switching to a method.

That's more like. What Matt said, like, based on the players prior, so, like, based on, okay, like, they average 30 pass attempts and, like, you know, if the only profit that we have posted is, like, they're passing touchdowns and that's a bit higher than average, like, bump up their passing sets to accommodate.

Yeah, so, you know, we're working on, like, figuring out that so that the early week Sims look better. But it's definitely, like, we get significantly more accurate as the props come up. And so, like, as that more data comes in that, like, significantly improves our projection accuracy. So, like, the later in the week, like, Thursday is when, like, a lot of, like, at least I've noticed on draft Kings, like, when a lot of the detailed player props, like, beyond just, like, Because early in the week, like, they'll put up, like, Lamar's rushing yards or something like that.

But

Matt: yeah,

Wil: as the week goes on, they'll give you Lamar's rushing attempts. And, you know, like, the, they'll get a lot more detailed in terms of, like, the number of players and what markets are offered. And that helps us considerably.

Jordan: Matt, you had mentioned, um, a change in the future here of basically exposing in the app. Like, Hey, we don't have props for this particular player. We're missing this prop for this particular player. I do think that'll be really useful for people that do like to tinker a little bit. Um, research a little bit target that research there.

In the meantime, are there types or categories of players or games or anything like that, that we kind of think maybe it is a little bit more useful to dig into for folks that do want to spend a little bit of extra time researching? I don't know if it's something like maybe by like salary or by even position or, or maybe like game script, like a spread or do we just kind of, is there any like, I don't know, like a breadcrumb of like, this might be something that you should actually kind of go look into.

Wil: Yeah, my first thought is with like the scrubs. Um, yeah, essentially like the, the players that, uh, either don't have props or have very illiquid props, uh, because like who's betting on them. Yeah. Um, those kinds of plays are like, they're, they're obviously going to be our weakest because they're the least informed markets.

Um, it's like, I, I feel like that's the most valuable. That's where my, my mind first goes with that kind of question.

Matt: Yeah, definitely agree. I mean, Showdown is going to be the big one where I think you'll, there's going to be more edge from that kind of thing where you. You know, like, you know, on a main slate changing somewhere from 2 points to 3 points doesn't change anything.

You're not going to get them regardless, but showdown that could be a pretty huge, you know, again, with the, the, the example of copy like that. He broke the slate, you know, he was in the winning lineup. He was 3 percent owned. Um, if you. Like, did research or like saw that news and pumped him up, you know, that's kind of where you can really add value, um, is if you have those, yeah, those, those takes on those players that don't have as much information.

Um, yeah, I mean, beyond that, it's like, I think the sources of, of edge beyond just like content, the built in stuff in SaberSim added value for users is like, you know, one, if you think that there are props that are wrong. Yeah. Yeah, um, you know, again, like, you know, if you're going to bet a prop, then you might as well and if for people that do bet props, like you should change their projection to, um, if you are think that the prop is wrong and it assuming it doesn't like chain, assuming it's not like betting on Thursday and then, um, You know, it changes by the time it's Sunday, but if it's close to, um, to kickoff and you're, you think the prop is wrong, then, and then also change the projection because you're kind of saying, this is my assumption about the player that I think is different than what Vegas is.

Thanks. Um, and then, uh, yeah, I mean, on the correlation, like, that kind of thing, they think there's probably something there, um, in terms of, like, maybe. You agree with, like, the mean prop projections, but you think that, like. In, uh, maybe you did research and you see that, like, a team when they're up by. 2 or more scores, they don't give any usage to.

X player, right? Yeah. Um, like this player is much more likely to just sit in a blowout, but this other player is much more likely to just keep playing. Um, you know, that, I think that's especially for something like NBA, like a really valuable, but I think probably even NFL, I haven't really done that type of research, but I would imagine that there's sort of the like range of outcome type.

Um, edges where you can see, like, obviously, you're going for a 99th percentile or more outcome with your lineup. And so, even if you think, like, this guy's mean projection is correct, I mean, um, like, you can see, like, okay, and in the games where, uh, the team is blowing out, um, this team is going to, like, rest all their starters and this team is just going to keep, like, passing the ball no matter what.

So, that's, that's kind of the other, uh, Area of that I would focus probably

Jordan: you had mentioned NBA. Is this something that we are interested in kind of taking this direction and applying it to other sports? I think NFL is obviously a little bit unique in just how massive the markets are. But, um, what are our thoughts for, like, Rolling this out to other sports.

Matt: Yeah. So definitely, I think we want to, in every sport, we want to try to use like sports book data, um, as much as we can, because again, there's like this value in the wisdom of the crowds. Um, NBA is especially tricky because they don't post props for. Nearly as many, I mean, NFL, there's props for every single player.

Um, at least like an editor type TD, right? Um, NBA, like they're often not going to post anything for like bench players. Um, other than kind of like the. Like expected bench players that are getting like 20 plus minutes. Um, and, uh, and then, you know, there's the late news where if there's a change in the starting lineup, they'll probably just pull the whole game and probably won't repost it, uh, or often won't repost it.

And so we do have to have a much better like internal. Model. I think we are still going to make a lot of improvements there where we're, I think, really going to focus on like those priors and making sure our baseline rates for all the players are accurate. I think like rookies, especially in like low players with low amounts of data, we have historically struggled a bit more with, uh, so that's probably going to be the focus so that in those situations where we don't have props, we have a really good baseline.

Uh, but yeah, I mean, in every sport, I mean, MLB, we, we, you know, Are not, we haven't quite gone the extent of NFL, but we are incorporating a lot of the, um. Sportsbook odds and and drops into the output, at least to kind of like, dial in the projections. Not as much as like. Fully 100 percent going that direction, but, you know, we're.

If anybody's been paying attention to that stuff, I think you probably would have noticed that we're much closer to. We don't have as many of these, like, huge. Edges versus the prop lines as we used to. But, yeah, I mean, I think every sport, that's the direction that we want to go. You know, moving forward.

Jordan: Am I, it feels also like our field lineups probably should have gotten at least like incrementally better here with this or does that make sense or does that? Yeah, I mean, we're going to be closer to, yeah,

Matt: I would say like in general, probably the The aggregate projections that the field is using are going to be pretty close to props because again, it's like the wisdom of the crowd, like, you know, there's going to be, it's not going to be a one to one and obviously like probably, um, more fishy, bad lineups are going to like, not care about the props and be like, Hey, I see a player that I know I like, or I follow.

You mean Dolphin's Fanatic

Wil: with his seven man Dolphin stack?

Matt: Yeah, exactly. Exactly. Um, But, uh, yeah, I think the fact that our projections are closer to the, like, overall field and the industry, um, are probably going to make the field that is more accurate. Just because, like, we're. Building the field, not just off of our projections, we do, like, adjust for other factors, but like, that isn't, um.

The projections that we have are going to strongly impact. Uh, the field that we're projecting, so. Yeah, I think it will be more accurate.

Jordan: What are some of the things that are still on the docket here for us? I know as soon as football starts, like we're kind of pushing out as much as we can and then NBA, you already mentioned it comes in really hot and we kind of have to transition pretty quickly.

Um, just some of the things that maybe we're still hoping to accomplish with this NFL season, or maybe even some like bigger kind of dream projects that we've seen as a result of doing this that we're like, Oh, you know, maybe even, you know, 2025 season, that would be really cool.

Wil: Yeah, so, for me, it's early week Sims.

That's my, like, number 1 focus at the moment is just, like, right now, like, I don't feel great about, like, what we post on Tuesday, Wednesday. And that's something that, like, you know, we're working on a lot of projects and I feel like we'll probably get them dialed in the next couple of days. But that's my, my main thing I'm still working on for football is, is making sure that, like, They're, they're not going to be as good because again, like, we're, we're leaning heavily into those props and without them, we're, we're missing that data.

Uh, but getting those to look a lot better is totally feasible and that's, that's what I'm, I'm hoping to knock out here soon.

Matt: Yeah, and not just early week, but also when there's late news and like, props get pulled for a game or something like that, we want to make sure that we're still able to update.

Our projections with, like, reasonable results, uh. And, and or, you know, players that don't have props, we want to make sure that we're kind of inferring the rest of their stats and using that historical data. I mean, obviously, historical data is also really important data to use. And we are, you know. We want to make sure that we're using that in conjunction or to fill in those holes that props are missing.

The other thing I think is, um. Uh, just adding, like, more markets to so, like, you know, like I said, we're, we're using the props. We're using all these all, um, team totals and and game. That's but I think, you know, there's so much that you can bet on and so many different markets that are provided and.

Probably continuously adding more, so I think, you know, again, some of those, like. Um, the, the, like all like prop stuff where it's not just like the mean or median, but you know, QB passing for three touchdowns and stuff like that. Obviously those are the one sided markets are more difficult to incorporate because it's hard to kind of remove the vague from that.

Um, the, the two sided ones are the ones I prefer to focus on, but there's, there's still like so much. Um, additional stuff out there that we can, and even like first half, uh, stuff or like quarter, you know, stuff about like, we can kind of improve even the game script, um, stuff with like those, you know, first half quarter level, you know, and at that point, you know, they are, um, Less liquid and so we need to be more cautious about, like, we don't want to go too far that we're, like, sacrificing some SIM integrity just to hit these props.

But yeah, the way that we incorporate them is sort of like, we're starting with the SIM and then kind of adjusting. So, you know, even if we have these, like, say, 1st half markets that are maybe adjusting what the SIM would do, like, we're running the SIMs and kind of adjusting. Um, after the fact in a lot of these cases, uh, so we kind of maintain that some integrity.

Jordan: Gotcha. Yeah, that's interesting. I might reveal some of my lack of understanding of, like, betting markets here by asking this, but how do we kind of reconcile 1? I guess the different quality. In different books, especially when they're in conflict. Um. But also like arbitrage is between even within a book where maybe like a certain line has gotten stale that isn't in line with the way that this even the sports book thinks that the game is going to go.

Like, maybe, you know, like 1, the prop is now out of date with the way that like, the book clearly thinks that the game is intended to go because of some news that changed. Like, how do we handle those kinds of things? Edge cases there,

Wil: so I think with regards to the 1st, 1, uh, I think there's like, 1 or 2 books that we, like, ignored for just like, being bad and not updating and that kind of stuff.

But in general, the books are pretty much in agreement, especially for, like, the major props, like, like, main players. Okay, they're like, they're not significantly different from 1 another. I mean, like, you know, some have, like, different vague and stuff like that, and you have to account for that. Um, and then with the other 1, we, we do.

Uh, to some extent, like, sort of, you know, we make sure that, like, the receptions, uh, like, the, some of the receptions props equal to the, the completions prop of the quarterback, like, that kind of things, like, making sure that those are in line. So it's, it's not like, You know, there's 23 receptions and 20 completion.

So, making sure that those line up, the yards line up, the touchdowns line up. Doing that, I think, handles a fair bit of your secondary concern there. Just, like, making sure that it all, like, flows into 1 concrete game script. I think that there, there may be, like, a smarter way to do that. Like, if you are able to determine, like, oh, it's because, like, this line is stale, we're, we're not doing that yet.

Um, right right now, we're, we're sort of taking them at. At, like, what they've posted, um, and then just, like, taking that and combining it and synthesizing it into, like, okay, now, here's what the aggregate targets are.

Matt: Yeah. Like I said before, we, you know, we do, we prioritize the, like, main, you know, money line game total spread, uh, more than anything else.

And, you know, those are going to be the ones that are trustworthy and are going to be. Um, not stale and so we're going to 1st, try to make sure that we're meeting those and then kind of like, cascade down. And then, yeah, everything will said, you know, is right as well. And we are, you know, we're aggregating.

So if there's different lines and odds on the different books, like, we're kind of aggregating the ones that ignoring certain ones that we don't trust, but, like. If there's differences there, we're well, um, I agree. I think there's probably room for improvement of like waiting certain books higher, just based on like accuracy and stuff.

I don't think the value of doing that for NFL is. Super high at this point, I mean, it probably is higher earlier. You know, if you're betting early, you know, there's going to be a lot more differences as we're getting close to the game starting. I doubt there's going to be a ton of. Significant differences between them, but, um.

You know, I think that's something that we'll look at and and maybe an area. Improvement of doing a smarter model of, of waiting different books and waiting different bets based on, um, their liquidity and all of that.

Jordan: 1 other question I had, and I actually meant to ask us as we were getting started here is just about some of the kind of the initial conversations that took place over this summer as we started to work on this.

What were, what were those like, were there particular, um, like issues that we saw when we were looking into last year that we were like frequently getting wrong on, I'm just kind of curious more about like the, the, um, the, the problems that we saw and that, and how this kind of, we came up with this as our, um, the solution.

Does that make sense?

Matt: Yeah. I mean, we, we have like, it's definitely not a new idea, right? Yeah. Using props as input. But I, we, I remember like three or four years ago, kind of. Trying to do that for NFL. I don't honestly remember why didn't work out. I think it was, um, maybe just the data that we were getting was, like, not being updated quickly enough, or just we didn't have a smart of like a framework for applying those.

Uh, but, you know, I think we had talked about it and, uh, I definitely think there were some concerns in the past about, like, oh, they're not like. Super efficient markets, like, we're not the props are beatable. And so if they're beatable, then we should, like, try to beat them rather than match them. And, you know.

Uh, I feel like I was pushing back on that line of thinking a bit in terms of, like, even if they're beatable, it's like, it's a lot of work and there's a lot of risk, uh, to trying to beat props and especially like, they're primarily beatable earlier and closing lines are just, you know, as the betting market has grown and grown, it's just like, it's harder and harder to beat closing lines for, um, you know, For really any, you know, any bets, it's certainly possible, but, um, it's not really worth it.

Um, you know, again, like we have a small team or a small company and I want, like, Will to be spending his time, like getting the SIM, which is like, that's our big value add, right? It's like this play by play SIM that no one, I think is even close to being able to like, you know, get. Replicate, like, the, the sophisticated this, um, of, uh, of, of that, um, framework.

So, focusing really on that and getting that dialed in feels like much better use of time. Um, but yeah, and just sort of, like, the common issues that we have throughout all of our sports of, like, The rookies, the players that had a lot of data, um, where like, that's where those props are so valuable. We don't have to worry about, uh, Oh, there's news that comes out that some X player is going to get limited before, like, there's no, there's no amount of data that we could have.

No matter how good our model is, we're not going to account for, for a coach saying the preference conference.

Jordan: Yeah.

Matt: That's where it's going to get limited. But the prop does tell us that because like immediately. Like through either through people betting it or just through the sports books, being good at this, they're going to like take the prop down and then put it back up with a lower number.

And then we just do that. And we don't have to like make a manual change and be following all that news constantly. Um, and we still have the ability to override and that is something like, well, we'd still do, um, checks of all projections every week. And we're still like looking at that and like paying really close attention to any overrides that we might need to make either through props being.

Wrong or not there or just like the SIM output doesn't match what the input should be for whatever reason. So we aren't paying attention that stuff. But, um, yeah, that's the biggest thing for me. It's just like any sort of information that is important to a game. That's going to, um, for the most part, be accounted for and makes it a lot easier for us.

So.

Jordan: Yeah, I think part of the reason that I was so excited about this is. When you have the, the play by play game Sims powering the lineups and, and then also the contest Sam, like you're so far ahead. You don't also need to handicap the market perfectly to be successful. And it's almost. Like you're almost taking on more risk than you need to there.

And even if you can, or like you're extremely skilled in doing that, the other side of this is, and frankly, I'm not like, that's not a strength of mine, but for the, the user that is. Now it's even easier for them to do that. Like I think kind of before we were kind of making that claim of like, yes, we have the playbook, we have a play by play simulation of the games.

We have a contest in that's helping you identify, but we're also kind of handicapping the market in a bunch of micro ways that even if they were really right, they were really hard to see what they were and they, you know, That led to you taking on a lot more risk. Now, um, you don't, you don't have that as much like I, it's almost like this, um, this is definitely not the right word, but it almost feels like a little more pure in a way of like, this is like, assuming this, like, set of information is kind of agreed upon about the games.

These are the most profitable lineups to play. And then if you disagree with the baseline assumptions, it's easier to see what they were. And also how to make your adjustments from there. Um,

Wil: yeah, I think that's the biggest thing. Like the getting people like the baseline and then letting people make their changes from knowing what the baseline is.

I think that's kind of like what, what got me like fully on board with this project and giving up my baby a little bit is like, it really is just like, I really like the approach and it also makes it a lot less stressful for me. Like, you know, I don't have to be like, okay, the, my, my decay rates and all of the things, you know, says that, uh, this guy's actually going to crush today.

And everyone says that he sucks. And so, like, then all Sunday, I'm like, sitting there, like, biting my nails to see if, like, he's actually good or not. Uh, now you can just change

Matt: his projection. You can do this on your own. Right. Right. Exactly.

Jordan: Yeah. I'm curious, like we talked NBA a little bit, baseball a little bit, but I mean, is there a home for some of this in some of the more niche sports?

Like I, well, it's already in MMA. Yeah. Okay.

Matt: Yeah, a little bit. Yeah, but we, we talked about this just this past week of like incorporating it a little bit more. Yeah, right. When it

Wil: comes to like decisions, because that's, that's like the biggest thing that I get question on that not to like, switch this to a UFC talk, but like, um, our decisions are decided in the SIM based on the outcome of the stats, um, and incorporating this approach a bit more will allow us to match, just match Vegas for decisions as well, uh, while still maintaining like, okay, they actually won.

Yeah. Um, it's like, that'll be really good. Uh, but yeah, like it is already like the concept of, um, you know, like in UFC, we use those like finish props as guidelines. So like, you know, like fighter wins, uh, in the first round, 11 percent of Sims. And if you looked at 11 percent of the Sims, you would see a first shot knockout for him.

Um, and so like, that's, uh, that's that concept is already applied elsewhere, but now it's in the main sports.

Jordan: Awesome. Cool. Well, I know we have to get out of here pretty quick. Um, maybe as a last question here, we've covered a lot of this kind of stuff throughout, but is there any other practical takeaways that you would leave our users with, um, sitting down to build on Sunday?

Um, anything else that this might change their, their process? One thing that jumps out to me, um, and maybe this is a bit of a cop out because I've recommended against doing this forever anyway. Um, If you are still aggregating projections, it seems even more of a waste of time to me now at this point. Um, essentially, like, knowing how we are kind of doing that for you in some ways, um, that's not something I've recommended almost ever.

So it does feel a little bit of a cop out, but that jumped out to me. Anything for for you guys have just like practical advice for people heading into this Sunday.

Matt: Yeah, I mean, I think I've talked about it. I mean, this is like, not exactly related to the model update, but it's sort of related to, like, the sim schedule stuff. And I mentioned it when I, like, announced it in Discord. But, like, I would really recommend to people to, like, build, like, early on Sunday. Um, I mean, obviously, like, in 1 way, that kind of contradicts what I was saying about, like, the closer we get to the start time, the more accurate stuff is, um, but I think because of the fact that we're not doing these, like, final Sims, like, I think just for NFL, especially, like, we have the vast majority of the news, um, for the slate early and obviously things can change.

Um, but it's really important, especially if you have. A like longer process, or if you want to do more research, like Do your builds, get everything kind of set up early and get lineups in like you would be happy playing and then there's kind of less stress about, uh, adapting to those later Sims. You can look at what's changing, you can rerun everything, but if you have everything already set up.

In, um, all of your rules, all of your, like, contest sims, all of your metrics, whatever, then it's just sort of like re running the build and re running the sim, or just re running the sim. Uh, and, uh, you know, I think just from a process standpoint, I think it's just really important to kind of like have everything in and ready.

Earlier, and then you're just having to, like, make smaller changes to adjust for, for new since they come in, rather than like, okay, I'm going to start my process 50 minutes before lock every week.

Jordan: Yeah, no, that's a good 1 and I think that's probably something that hopefully barring another Evan Ingram this week.

We'll have another opportunity to kind of practice on, but what I was prepared to do this week was exactly as you described. I think I had my initial builds. Done like an hour and 15 minutes before lock. And I was just going to Sim contest summit again, if there was subsequent Sims with no significant projected changes, obviously we ended up getting the Evan Ingram news, but also very cool.

And a cool case study for that is. If I remember correctly, we just ran one additional SIM after that news broke, and we kind of had updated to that one, like in the past, I think a lot of our users are probably used to the idea of like, there's kind of that initial SIM that runs and then sometimes we look at the results of that, and then we have to tweak a little bit more and then maybe potentially, not to say that will never happen, but knowing where some of our information is coming from now, I would assume those kind of initial, Changes for news will be a little sharper.

Matt: Yeah. And just to clarify, like the process there is like, we have it set up right now so that if folks basically pull props, which usually happens, if there's an injury like that, they'll like, yeah, initially they'll just pull everything down and then it'll be like five, 10 minutes later where they'll kind of start posting, um, everything again.

And so in those cases, we're just going to wait. So basically like if there were props for a game and then they get pulled, we're not going to run. new sims like guessing and then run another sim once they're pulled we'll just sort of wait for things to settle i mean assuming we're not like right before a lock um this don't don't take what i'm saying for like nba um yeah but like uh you know for nfl assuming it's not like okay there's 10 minutes before lock and someone got ruled out like i mean in that case maybe we'll just like mark them out and let everybody it just depends on the timing of everything but yeah in general um We're going to wait for the props to get posted again.

And, you know, the team is, is actively monitoring too. It's like, we're always going to be like looking at the stuff when the Ingram stuff news came out, we were like, okay, let's like, See what the data is like make overrides if we need to. I think maybe we initially did an override for strange as backup.

And, um, I think actually in that case, when they posted props for strange, it ended up being like exactly what we had already done as the override. So we didn't. even need to like do another sim on top of that. Sometimes, you know, especially in those cases, you know, the initial props that the books put up for those like new players might not be very sharp.

And so that's probably a situation where we might run a couple sims because it's like, okay, they're posting this new player, this new profit, like 20 and a half yards. And then he ends up getting steamed up to 30 and we'll want to account for that, that difference. Um, so. You know, again, like you're right that like, hopefully we don't have to run a time, but in those cases where it's like new, like big news, like we might try to dial in, but from a process perspective, I definitely would recommend like.

You know, quick swap the out player out, get an initial build in with like the updated news and then kind of just monitor and rerun if you can. And, and, um, you know, just, you always want to be in a spot where like you're comfortable with the lineups that you have uploaded to, to the game. The sites, I never feel like, okay, if lock just suddenly happened, or if I lost internet right now, I would be completely screwed.

You always want to be in a situation where you're like, okay, it wouldn't be like completely screwed. I would,

Jordan: that is a really good way to put it. Yeah. I, I, that is a great line. I am going to steal that. But why I've never just said it like that before in any other video, but yeah, just at any point you're not screwed.

If your internet went out is a very good way to think about your, your lead up to lock. So I like it. Uh, Will, did you have a practical takeaway you wanted to give everyone? Are you still a push button, print money? Yeah,

Wil: I am literally, that was the exact thing I was about to say. All I do is just trust the Sims.

Um, I'm a bit biased because I worked on them. So yeah, no, I, yeah, I, I, my process is the most boring process in the world. Um, I make a few tweaks and like X out players that I don't want to play. Uh, and then I build, run a contest sim and upload.

Jordan: Which, uh, which, which players do you not want to play this week?

Cleveland's offense. All right.

Wil: Well, there's the tout show for the day.

Jordan: Yeah, exactly. I said, we'll leave everyone with the, with the take, uh, fade Cleveland. Cool. Sounds good. Um, well, I know we need to run, so, um, we'll go ahead and cap things off there for anybody that has, uh, stuck it out with us here. We do have a free five day trial sabersim.

com. If you want to come check us out. For a free five days, uh, for any of our existing members, as always conversation continues in discord. If there are questions, comments, uh, any thoughts you have that you wanted to share for the three of us, you can pop those into the discord server. Uh, but Matt will thanks for hopping on and chatting with me and we'll see everybody later.

Thanks.

‍