Find Jobs
Hire Freelancers

Process 100 million lines of domains and output the UNIQUE lines.

$30-250 USD

Stengt
Lagt ut over 9 år siden

$30-250 USD

Betalt ved levering
We have a list of domains about 100,000,000 in total, I imagine about 50% are duplicates. We need to process the entire list and remove the duplicates. The final output should be a list of UNIQUE domains.. I have been processing using EMeditor for the past 48 hours on a an i7 PC with 16GB of RAM, and it's still no where near finished. We need some massive power to process this. Please do not bid unless you have worked with data of this size before. Thanks!
Prosjekt-ID: 6841247

Om prosjektet

42 forslag
Eksternt prosjekt
Aktiv 9 år siden

Ønsker du å tjene penger?

Fordeler med budgivning på Freelancer

Angi budsjettet og tidsrammen
Få betalt for arbeidet ditt
Skisser forslaget ditt
Det er gratis å registrere seg og by på jobber
42 frilansere byr i gjennomsnitt $147 USD for denne jobben
Brukeravatar
Hello, This is vaishali from \"Hire WordPress Experts\" and I am here to help you with Process 100 million lines of domains and output the UNIQUE lines.. We have gone through the information provided by you. and I can assure you that we are expert with Data Processing, Excel, Microsoft SQL Server, MySQL, PHP, and I will make your project a reality. Surely, We are ready to help you with process the entire list and remove the duplicates. Yes, The final output will be a list of UNIQUE domains.. Yes, I have understood your requirement. We will assure you that we can easily do it. Give us an opportunity to prove our self. I can promise following 1. You will have exceptional time working 2. We do not hesitate if specifications change for good. 3. The project will be completed in far less than 25 days and you receive free support for up to 25 days. 4. We will be available on sky-pe 5. We do not take any upfront payments Please select us right away, so that we can get started without wasting a day! Thanks P.S. Be assured you will have a nice time, and this is a placeholder vaishali Business Analyst
$99 USD om 25 dager
4,9 (587 omtaler)
8,9
8,9
Brukeravatar
Hi ready to work with you if you think you need a good worker then you can hire me...............Thanks.
$277 USD om 3 dager
4,9 (483 omtaler)
8,2
8,2
Brukeravatar
I can remove duplicates from 100 million lines of domains list and output the unique lines in sql or text format. I will complete this work in 2 days. Looking for your reply to start this work immediately.
$79 USD om 2 dager
5,0 (1337 omtaler)
8,3
8,3
Brukeravatar
hi I'd like to help you on this. I have had to accomplish similar goals before and am ready to provide you with a custom, repeatable process in the form of a bespoke application that will run utilising all the power your pc has (all cores, parallel) and cleanse the domain list in the form of a bespoke .net application. Also I will clean the data for you if you provide the file. regards Janos
$389 USD om 3 dager
5,0 (90 omtaler)
7,9
7,9
Brukeravatar
I have experience to work with 20 million rows, I have dedicated server located in texas, I can process your data within 24 hours, I have many alternative way to process data. I will not demand any penny if I failed to do so ( 0.00001% possibility of being getting failed )
$50 USD om 1 dag
5,0 (249 omtaler)
7,3
7,3
Brukeravatar
Hello, removing the duplicates won't take more than 48 hours. Regards.
$200 USD om 3 dager
5,0 (89 omtaler)
7,1
7,1
Brukeravatar
Hello Sir, I can create this list for you but it will need different approach than you were using. Check my profile to see that people who worked with me are extremely satisfied with results and speed. I have 100% completion rate and I can start right away. Best regards, Dusan
$99 USD om 3 dager
5,0 (153 omtaler)
7,2
7,2
Brukeravatar
Hello I can do it Please provide me the domains list. ......................................................... Best Regards Bill Lee
$100 USD om 3 dager
5,0 (61 omtaler)
6,9
6,9
Brukeravatar
A proposal has not yet been provided
$200 USD om 3 dager
5,0 (61 omtaler)
6,5
6,5
Brukeravatar
Hi there, I'm exert in Database Management. I can do this. Please PM me for further discuss. Thank you, FARZANA PINKY.
$222 USD om 2 dager
4,8 (118 omtaler)
6,3
6,3
Brukeravatar
Hi, I'm an expert in database and data processing with very good feedback and completion rate. I'm very interesting in your project and willing to do it for $100. I used to process big databases up to 24 millions records and believe I can do your project in 3 days. My method is importing 100 million lines to a database, using powerful function of the database to remove duplicates (much faster than text editor like EMeditor), and then exporting unique records to the result file in original format. Best, svteam.
$100 USD om 3 dager
5,0 (62 omtaler)
6,2
6,2
Brukeravatar
What is the file format of the data ? I propose a custom 3 step approach: 1. filter data, separate files for each letter; 2. sort alphabetically each fille; 3. process each file and keep unique records. Filtering is fast, sorting takes some time, processing is much faster on sorted data. Can you please send me a small sample file ?
$149 USD om 5 dager
5,0 (216 omtaler)
5,7
5,7
Brukeravatar
Hi, I am experienced systems administrator, I worked for a company processing large amounts of similar data (traffic logs for telco) - I performed analysis and reporting of that data in both databases and flat files. What's needed in this project is to analyze input file(s) and transforming the data to the form it is optimized to processing. I can handle this task and deliver the required, unique domain list (along with number of occurences in the input file(s) if wished for). I can also deliver the procedure for analysing the data in this case.
$80 USD om 3 dager
4,8 (15 omtaler)
4,4
4,4
Brukeravatar
Hello, I can help you with your project. If it's OK with you, could you share the file containing 100M domains? I can give you the exact number how many unique domains. Thank you.
$222 USD om 2 dager
4,6 (2 omtaler)
3,4
3,4
Brukeravatar
Software developer since 2000, specializing in Visual Basic, SQL Server, Crystal Reports, and VBA for MS Office. please send a sample of your list of domains
$100 USD om 3 dager
5,0 (4 omtaler)
3,3
3,3
Brukeravatar
Just tell me that where these "100 million lines of domains" resides. Either these are in a text file or any other format (please specify). I will make a program / macro that will read this file and remove the duplicate domain names. More discussions will be after you accept my bid. Thanks.
$250 USD om 3 dager
4,9 (13 omtaler)
3,5
3,5
Brukeravatar
Hi, expert web/data scraper here with over 17 years experience in programming and RDBMS - please see my reviews. I'm using Perl for this kind of jobs. I'm able to finish fast.
$244 USD om 3 dager
5,0 (5 omtaler)
3,5
3,5
Brukeravatar
Hi! I am interested in your project. I am a software developer and I am working in data + web analysis so I strongly believe that my abilities fit to your requirements. I look forward to working with you!
$150 USD om 3 dager
4,8 (1 omtale)
3,1
3,1
Brukeravatar
Hi I worked on US Health Care dataset of 20 million rows. 100 million rows not yet. If you could let me work on the file now, I can confirm if my solution work prior to award. If not successful, no charge please. KC
$166 USD om 3 dager
5,0 (6 omtaler)
3,1
3,1
Brukeravatar
Greetings! Software developer with expert knowlege of data processing, at your service. Let me code a script for you that will go through your input and output a list of unique domains. I'm ready to start working now, let's get the job done! Thank you.
$30 USD om 1 dag
5,0 (2 omtaler)
3,0
3,0

Om klienten

AUSTRALIAs flagg
Melbourne, Australia
5,0
216
Betalingsmetode bekreftet
Medlem siden apr. 20, 2012

Klientbekreftelse

Takk! Vi har sendt deg en lenke for at du skal kunne kreve din gratis kreditt.
Noe gikk galt. Vær så snill, prøv på nytt.
Registrerte brukere Publiserte jobber
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Forhåndsvisning innlasting
Tillatelse gitt for geolokalisering.
Påloggingsøkten din er utløpt og du har blitt logget ut. Logg på igjen.