Bruce Appsumo - Customer Support Thread

So uh automation number two, now that we have all the data that we need in a sheet.

So we'll call this one, number one.

I, I then want to build a second automation to scrape what it's gonna do is gonna navigate to each one of these URL S and scrape the data that is needed off those, off those sheets.

So what I'll do is I'll click, add new automation capture steps and same process log in to linkedin.

Uh Here we go.

No.

Are we in now? We're not.

OK.

Here we are.

All right.

Cool.

Log in linkedin.

And then I'm gonna navigate directly to each one of these URL S and then I'm gonna scrape whatever you need off of here.

Now, what you're gonna want is a second um a second tab with the highlights or with the names of headers for each of what you want.

Scraped, whether it's again, company name, location, uh the number of applicants, uh full time, what type of position employees, whatever you want and a, a direct URL apply button link.

So whatever you want.

So you're gonna want that in here.

So let me just put that together.

So please l so that was positive ones.

All right, cool.

So this for, I have this for demo purposes and just for this video purpose to make it short.

I'm not sure exactly everything that you need here.

But um I did get company RRL, company name, company location, number of employees, job type coins with company and the direct link.

So I have all that there.

So what I'm then gonna do is I'll scrape for each one of those.

So the scrape list or actually I'm gonna scrape text in this one.

Great text captured.

It looks like I don't need a URL for this one because it won't be a URL.

So I'm just gonna just be company name.

That's my next one, company location.

Well, again, I want scrape text, not scrape lists, it's captured and then number of employees scrape text, got that and number or job type.

So you get my point.

We go text um this guy here and then I want the direct URL.

So I didn't get the URL on that one.

No URL on that one either.

So I'm not sure if you need that or not, but if a URL is included, then it, it'll pop up.

But since it's not popping up, there's only a text included.

So I have everything that I need there.

Um I don't need to add in.

I don't need to account for PAG multiple chain pages on this because it's just one page.

I'm gonna end the automation because I have all my steps then come in here and clean up.

So if you noticed I had, I did some, I double clicked when I was trying to log in, I wanna clean that up.

Uh sign in, come to add a delay.

So let it load and I'm going to navigate directly.

Oh, I need to attach my sheet because I want to have it navigate to each one of these.

So the way I do that is I'm gonna set up trigger loop, trigger, set up sheet this time, add my worksheet.

Uh And I'm gonna have it pulled from scrape test number one or just the scrape test alone.

Um I wanted to start with row two and I wanted to loop through three times and you can schedule that whenever you want.

I'm gonna mess with my loose once I'm done editing my my sheet here.

So what I'm gonna have to do, what I'm going to do is add in a direct a navigate step.

And what I want to do is navigate directly to need to report a bug there.

Um I needed to search company me or it's not finding it.

So let me just let me test my connection here.

Uh It's not connected.

Ok.

It might be a better report.

It's great to looks good.

Ok.

So connected now.

So now I can wanna add that delay or navigate that Right.

Yeah.

It's just not, it's not connecting, I mean, connecting people instead.

See if that works good.

Toast already.

Have a navigate stuff in here.

There we go.

Well, how's my URL in here? So kind and? Ok

Hey, Bruce, this is Vince getting back to you here.

I appreciate you sending that in.

So I can absolutely help you with this.

Um The video is a little bit tough to understand like really, really just everything that you were trying to scrape.

And so I just want to give some clarity on some easier ways to scrape off of linkedin, the scraping in general.

And then, um and then I can offer some guidance if you don't wanna do it that way and then do some things to pay attention to.

So here's, here's what I would recommend.

Uh I'm gonna open up linked in here and so you might want to split this into a few different automation that way.

It's just cleaner.

There's less chance for it to, to be stopped and, and have any errors.

Let me log in really quick.

OK? I'm logged in.

And then what I wanna go to is let me actually just repeat what you were doing skin.

Let me grab the filters that you were using key words for software developer.

Got a team down.

Yeah.

All right.

And so what I would recommend is, I mean, do what you normally do like you just did with the um So do what you wanna do as you normally do.

You click enter here, open up the search bar for the filter to search.

Oh I went through and then I want my keywords here.

I'm not sure if I got what you got, what you got you in jobs and jobs.

OK? And then United Kingdom and extra clicks here don't matter because what I'm gonna do is I'm gonna delete some of these steps and just add in a direct, direct URL um to the page that I want to scrape from.

OK.

So then what I would do from here is capture some scrape steps.

So if you want like if you want this information, for example, this is one way to build it.

I think it would be cleaner.

Um There was a lot of audio in your, in your video.

So I didn't know what the whole outcome.

I don't know the exact outcome of what you're trying to achieve.

But if it were me, what I'd be doing is I would scrape list and I would be grabbing the names of the company so that I can choose either text.

I can grab the text and the URL name and direct URL.

I'll grab URL first and I'll be in column A and I'm gonna grab then the name.

So I know which company it is and then I'll grab text and then what else do you want here? Oh, we want a location.

Let me grab, I don't need your order.

All of this.

Here we go.

We got location.

So I have the URL, the name and then the location here and I just need to text you that since the new er L was not offered.

And this guy, OK, then from here, I'm not sure exactly what you want anything else, but what I would do is I would stop the automation.

Now, this would be automation number one.

And so what this will be is uh this will, I'm gonna come in here.

This is gonna scrape all the direct URL S to uh a spreadsheet.

And then I'm gonna attach a build a second automation to loop through each or each URL and scrape the data I need to each URL.

So this one log in sign in search, I'm actually just going to build a delay step here.

Uh And what's I and, and then I also, what I want is the direct URL of um of that page.

So what I'm gonna do is log back in here.

Give me a sec.

OK? So log back in and in a separate browser.

I'm gonna repeat for jobs.

Is that not opening up a second wars? There we go.

And jobs.

OK? United Kingdom.

OK.

So I have and, and the separate, this is not recording, this is not Chromium.

I'm gonna copy this URL.

So it can be in any URL, any um browser Safari Foxfire doesn't matter chrome.

And then I'm gonna add a direct navigate step to go directly to that page.

That way it's cleaner.

I don't need all these steps which is click search enter and these are all places where it could fail.

Now, if you do need, require these steps or if something's want, if you want built in, we do have an allow air feature.

So I would recommend clicking, allow er on any step that's not required to achieve the outcome of your automation.

Enter.

As you can see here, there's a lot of steps here.

Jobs.

United States.

And really, I just need to get to my re steps, ok? I just need to get to my scrapes up.

So I have my scrape link, my URL, the name and then the I think the location, then I'm gonna send this to a Google sheet.

OK? Let me just choose.

I'll just do spear tests here.

Let me edit to make sure I have three columns with headers in it.

Great test you go.

So we have the company URL, no company name and the location.

I want them all bolded.

Ok, cool.

And then I'm gonna send, I already have it connected.

So let me just check my sheet connection.

Yeah, make sure it has my headers.

That looks good.

Step eight is link's name.

OK? Cool.

And then I'm just gonna run this just scrape all that data.

I gonna trigger this to run and should be scraping data to this page here.

I'll log into my linkedin.

We have a single sign on feature too.

So remember if you log in so you don't have to log in every time, which will help reduce any errors or pro log in prevention.

Cool.

So you can see the page here not be directed to the page a little bit cleaner.

Let me see.

It's navigating through some of those scrape steps.

Ok? So it then scraped the company URL company name and location.

Now I only grabbed the that I only grabbed the uh it only grabbed the first few that were visible in this screen.

So now what I wanna do is add in a scroll step.

Let me just add a delay to let it finished rendering and then I'm gonna add in a custom scroll step and then I can choose how many, how many seconds I wanted to scroll for.

So we'll say I can test it in normal browser.

Oh, and we also have pagination in here too.

OK? So good to know.

We're gonna need to click.

We're gonna need to build ation as well to get all the pages if you want.

So let me just say uh 15 seconds, I want to scroll down for 15 seconds.

And then I'm gonna add in a step, a custom click stuff because I didn't grab it, which is gonna be looping through and I'm gonna need to attach a loop through.

Yeah, Ed.

Yeah, I want to grab this button here actually.

Uh yeah, grab the ex pat and selector.

So and the cool.

So what this is gonna do it will log in uh navigate directly to that page.

Scroll down 15 seconds and scrape all of everything that I need.

Company link, company name and then company location and then it's going to click on the next page and then I wanna now build in a loop trigger.

I'm not gonna attach any sheets.

So I'm gonna skip that and I want this to loop through.

I have no idea how many times here.

Let's see.

You can choose how many times you want this to loop.

You know, we could say loop through 20 times, but which means it's gonna go to 20 different pages.

You can see there's 17,000 results on here.

So I'm just gonna choose loop through three times for this for time purposes.

And then we can schedule this to run off the desktop.

Whenever whenever you want, I have my loop, set my loop steps and loop setting built in here.

So what I don't want it to loop back and log in and I'm gonna start the loop at, at step nine.

So as soon as you go, it clicks the next page, it's gonna scroll down, grab the links and then click next.

It'll do that three times.

So I'm gonna run this really quick.

Um Now our scroll down step, there is a bug with our scroll down step, which is under, which is now being developed right now.

So by the time you get this video and try this out, it should be fixed and it should be updated, but it's likely not gonna scroll down, but we need to scroll down in order to grab all of the companies off that page.

And if there's any errors we can and work from there, I'm thinking it's gonna air at the click step which is a click next page.

Um And I can play with the selectors and the X pass, but it might be better just to recapture steps.

Yeah, aired, aired at 13 as I thought it would.

So in that sense, here's what I would do.

I would just take this recapture steps here and navigate directly to that page that you need.

Or actually let's go to linkedin first log in, but it captures a login.

OK? I am in linkedin and I'm just gonna navigate directly to that page copy RL and then grab my scrapes ups.

So we're looking at great list.

It's pretty much repeating this.

So I'm gonna click URL Craig list, clicking text scrape list location, which will be text only.

So I have all those lined up here, then I'm gonna have it scroll down and I want to capture this next button.

Ok.

Good.

To go here.

All right.

So uh it's gonna log in and I'm going to add in a delay step.

It was important add in a direct, navigate, direct URL set to navigate directly to that page.

And then another delay allow it to finish rendering.

And then I'm gonna add in a scroll down, which I'm confident right now is not gonna work, but it will here soon.

We're updating it right now, let's say 15 seconds and then it's gonna scrape all this data.

It's gonna click and I want to send two sheets which should already be connected.

Awesome.

And I'm gonna set up my trigger, trigger should also be connected.

So I, all I did was recapture steps.

OK? Cool.

And let's test this out.

Keep in mind I'm not confident it's gonna scroll down.

This should at least capture the next page or that uh need to deal with the pagination on the page now since it, there we go.

So it did capture.

So it's gonna loop, it's gonna loop through three times.

Um So remember it's only gonna grab what it's facing here so I can see it's not scrolling.

So that's the one thing that we do need to update, which we will update as soon as possible here.

Yeah, and it's good.

I'm just gonna loop through three times.

All right, cool.

So we got more data.

Um But we did not grab all the data off of each page because the scroll, scroll down didn't work right now.

So that is automation number one.

So I'm gonna stop the video now and then I'll show you automation number two