Choosing the Right Programming Language for a Career: Python, Go, or Java?
In today’s competitive job market, choosing the right programming language for a career can be a daunting task. With so many options available, it’s natural to feel overwhelmed. In this article, we’ll delve into the world of Python, Go, and Java, and use data to speak to their popularity and job prospects.
Getting the Data
To analyze the market demand for these three languages, we turned to our good partners at Reptiles and Pull Hook Net. We grabbed market data from their websites and analyzed the results. Our goal was to provide a clear picture of the current job market for each language.
Understanding the Analysis
Before we dive into the analysis, it’s essential to understand the data collection process. We used the Chrome DevTools to capture network packets and identify the request mode. In this case, we noticed that the request mode was a POST request, rather than a GET request. This is crucial information, as it affects the way we collect data.
Climbing the Page
To collect data, we used the requests library in Python to send a POST request to the Lagou website. However, we encountered an anti-climbing mechanism that blocked our access after a few requests. To overcome this, we used a GET request to retrieve the cookies and then used the requests.session method to send the POST request.
Code Snippet
Here’s the code snippet that demonstrates how we collected the data:
import requests
import re
from bs4 import BeautifulSoup
import pandas as pd
import xlwt
import random
import time
base_url = 'https://www.lagou.com/jobs/positionAjax.json?needAddtionalResult=false'
lagou_list = []
def get_lagou(keyword, page_count):
for i in range(page_count):
# Cookie_dict = dict()
# S = requests.session()
# S.cookies.update(cookie_dict)
# Print(cookie_dict)
Myheaders = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.131 Safari/537.36','Referer': 'https://www.lagou.com/jobs/list_java?labelWords=&fromSearch=true&suginput='}
time.sleep(5)
session = requests.session() # get sessions
session.headers.update(Myheaders) # update the header information, cookies maybe
session.get("https://www.lagou.com/jobs/list_python?city=%E5%85%A8%E5%9B%BD&cl=false&fromSearch=true&labelWords=&suginput=")
try:
from_data = {'First': 'false','Pn': str(i + 1),'Kd': keyword}
print('%s are crawling on page posts%s'% (keyword, str(i + 1)))
response = session.post(base_url, headers = Myheaders, data = from_data)
response.encoding = 'utf-8'
info_list = response.json() ['content'] ['positionResult'] ['result']
companyIds_list = []
for job in info_list:
information = []
information.append(keyword) # positions corresponding to the ID
information.append(job ['city']) # positions corresponding to the city
information.append(job ['companyFullName']) # Company Name
information.append(job ['companySize']) # size of the company
information.append(job ['companyLabelList']) # benefits
information.append(job ['district']) # workplace
information.append(job ['education']) # academic requirements
information.append(job ['firstType']) # Type of
information.append(job ['positionName']) # Job Title
information.append(job ['salary']) # Payroll
information.append(job ['workYear']) # working life
information.append(job ['financeStage']) # development stage company
information.append(job ['skillLables']) # Skills
lagou_list.append(information)
companyIds_list.append(job ['companyId'])
# Print(companyIds_list)
# CompanyIds_str = '%2C'.join(str(s) for s in companyIds_list)
# Print(CompanyIds_str)
# Get_url = 'https://Www.lagou.com/c/approve.json companyIds=' + companyIds_str
# Print(Get_url)
# Res = requests.get(Get_url, headers = Myheaders)
# Get_cookies = res.cookies.get_dict()
except Exception as e:
print('procedural error', e)
continue
return lagou_list
def main():
info_result = []
title = ['job type', 'City', 'company name', 'company size', 'welfare', 'place of work', 'academic requirements', 'type of work', 'job title', 'pay', 'working life', 'the company\'s development stage', 'skill requirements']
info_result.append(title)
# Grab python language information
lagou_list_python = get_lagou('python', 20)
# Crawling java language information
lagou_list_java = get_lagou('java', 20)
# Language information go crawl
lagou_list_go = get_lagou('go', 20)
info_result.extend(lagou_list_python)
info_result.extend(lagou_list_java)
info_result.extend(lagou_list_go)
# Create a workbook, namely excel
workbook = xlwt.Workbook(encoding='utf_8_sig')
# Create sheet, the second parameter is used to confirm whether the same cell unit may reset the value
worksheet = workbook.add_sheet('lagou', cell_overwrite_ok=True)
for i, row in enumerate(info_result):
for j, col in enumerate(row):
worksheet.write(i, j, col)
workbook.save('lagou.xls')
if __name__ == '__main__':
main()
Analysis of the Data
We analyzed the data collected from the Lagou website and created three bar charts to visualize the job market for each language.
1. Salary Comparison
The first chart shows the salary distribution for each language. We can see that the Go language has a relatively high salary, while the Python language has a more even distribution. The Java language has a lower salary, but it’s still a popular choice.
2. Position Demand
The second chart shows the job demand for each language in major urban centers. We can see that the traditional deep north of Guangzhou has a high job demand, while Hangzhou and Chengdu are also rising stars.
3. Academic Requirements
The third chart shows the academic requirements for each language. We can see that most jobs require an undergraduate degree, but a graduate degree is preferred for higher salaries.
Conclusion
In conclusion, the data suggests that the Go language has a relatively high salary and a strong job market, while the Python language has a more even distribution and a higher demand. The Java language has a lower salary, but it’s still a popular choice. When choosing a programming language for a career, it’s essential to consider the job market, salary, and academic requirements.