Python编程轻松抓取新东方课程,小白也能学会的实用教程

在互联网时代,学习资源丰富多样,新东方作为国内知名的教育平台,提供了大量的课程资源。对于想要自学或者提升自己的朋友来说,掌握如何抓取这些课程内容是一项非常有用的技能。今天,我就来给大家分享一个Python编程轻松抓取新东方课程的实用教程,即使你是编程小白,也能轻松学会。
准备工作
在开始之前,我们需要做一些准备工作:
安装Python
:如果你还没有安装Python,请前往Python官方网站下载并安装。
安装相关库
:我们需要使用
requests
和
BeautifulSoup
这两个库来发送网络请求和解析HTML。你可以使用pip命令来安装它们:
pip install requests
pip install beautifulsoup4
了解课程结构
:在新东方网站上浏览课程页面,了解课程的基本结构,这有助于我们后续编写代码。
发送网络请求
首先,我们需要编写代码来发送网络请求,获取课程页面内容。
import requests

def get_course_page(url):
try:
response = requests.get(url)
response.raise_for_status() # 检查请求是否成功
return response.text
except requests.RequestException as e:
print(f”请求失败:{e}”)
return None

# 示例:获取某个课程的页面内容
course_url = “https://www.xdf.cn/kc/1234567890/”
course_page = get_course_page(course_url)
解析HTML内容
获取到课程页面内容后,我们需要解析HTML,提取出我们感兴趣的信息。
from bs4 import BeautifulSoup

def parse_course_page(course_page):
soup = BeautifulSoup(course_page, ‘html.parser’)
# 假设课程名称在标签中<br /> title = soup.find(‘title’).get_text()<br /> # 假设课程描述在class为”course-intro”的div标签中<br /> intro = soup.find(class_=’course-intro’).get_text()<br /> return title, intro</p> <p># 示例:解析课程页面内容<br /> title, intro = parse_course_page(course_page)<br /> print(f”课程名称:{title}”)<br /> print(f”课程描述:{intro}”)<br /> 保存课程信息<br /> 最后,我们将解析出的课程信息保存到文件中。<br /> def save_course_info(title, intro, filename=”courses.txt”):<br /> with open(filename, “a”, encoding=”utf-8″) as f:<br /> f.write(f”课程名称:{title}\n”)<br /> f.write(f”课程描述:{intro}\n”)<br /> f.write(“-” * 20 + “\n”)</p> <p># 示例:保存课程信息<br /> save_course_info(title, intro)<br /> 总结<br /> 通过以上步骤,我们就可以轻松地抓取新东方课程的信息了。当然,这只是个简单的例子,实际应用中可能需要处理更多的细节,比如分页、课程列表的抓取等。不过,掌握了这个基础方法,相信你已经能够应对更多的抓取任务了。<br /> 希望这个教程能帮助你入门Python编程和网络数据抓取。在学习的过程中,遇到问题不要气馁,多查阅资料,多实践,你一定会越来越厉害的!</p> </div> <div class="wp-block-group has-global-padding is-layout-constrained wp-block-group-is-layout-constrained" style="margin-top:var(--wp--preset--spacing--40);padding-bottom:var(--wp--preset--spacing--50)"> <div class="wp-block-group has-global-padding is-layout-constrained wp-block-group-is-layout-constrained"> <div style="height:var(--wp--preset--spacing--40)" aria-hidden="true" class="wp-block-spacer"> </div> <hr class="wp-block-separator has-text-color has-contrast-3-color has-alpha-channel-opacity has-contrast-3-background-color has-background is-style-wide" style="margin-bottom:var(--wp--preset--spacing--40)"/> <nav class="wp-block-group is-content-justification-space-between is-nowrap is-layout-flex wp-container-core-group-is-layout-2b923ac6 wp-block-group-is-layout-flex" style="padding-top:var(--wp--preset--spacing--40);padding-bottom:var(--wp--preset--spacing--40)" aria-label="Posts"> <div class="post-navigation-link-previous wp-block-post-navigation-link"><span class="wp-block-post-navigation-link__arrow-previous is-arrow-arrow" aria-hidden="true">←</span><a href="https://ziliao.tiaomu.com/116407299/" rel="prev"><span class="post-navigation-link__label">Previous: </span> <span class="post-navigation-link__title">揭秘电力科技部工作亮点:创新实践与安全效益双赢之道</span></a></div> <div class="post-navigation-link-next wp-block-post-navigation-link"><a href="https://ziliao.tiaomu.com/116407301/" rel="next"><span class="post-navigation-link__label">Next: </span> <span class="post-navigation-link__title">告别卡顿!CPU转GPU渲染,揭秘加速游戏与视频的秘诀</span></a><span class="wp-block-post-navigation-link__arrow-next is-arrow-arrow" aria-hidden="true">→</span></div> </nav> </div> </div> </main> <footer class="wp-block-template-part"> <div class="wp-block-group has-global-padding is-layout-constrained wp-block-group-is-layout-constrained" style="padding-top:var(--wp--preset--spacing--50);padding-bottom:var(--wp--preset--spacing--50)"> <div class="wp-block-columns alignwide is-layout-flex wp-container-core-columns-is-layout-3a88641f wp-block-columns-is-layout-flex"> <div class="wp-block-column is-layout-flow wp-block-column-is-layout-flow" style="flex-basis:30%"> <div class="wp-block-group is-vertical is-layout-flex wp-container-core-group-is-layout-831b2db5 wp-block-group-is-layout-flex"> <p class="wp-block-site-title has-medium-font-size"><a href="https://ziliao.tiaomu.com" target="_self" rel="home">资料库 – 中文知识库</a></p> </div> </div> <div class="wp-block-column is-layout-flow wp-block-column-is-layout-flow" style="flex-basis:20%"> </div> <div class="wp-block-column is-layout-flow wp-block-column-is-layout-flow" style="flex-basis:50%"> <div class="wp-block-group is-content-justification-space-between is-layout-flex wp-container-core-group-is-layout-cf54d0a6 wp-block-group-is-layout-flex"> <div class="wp-block-group is-vertical is-content-justification-stretch is-layout-flex wp-container-core-group-is-layout-d8dce8eb wp-block-group-is-layout-flex"> <h2 class="wp-block-heading has-medium-font-size has-body-font-family" style="font-style:normal;font-weight:600">About</h2> <div class="wp-block-group is-vertical is-layout-flex wp-container-core-group-is-layout-bce0c455 wp-block-group-is-layout-flex"> <nav style="font-style:normal;font-weight:400" class="has-small-font-size is-vertical wp-block-navigation is-layout-flex wp-container-core-navigation-is-layout-bce0c455 wp-block-navigation-is-layout-flex" aria-label="About"><ul style="font-style:normal;font-weight:400" class="wp-block-navigation__container has-small-font-size is-vertical wp-block-navigation" aria-label="About"><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Team</span></a></li><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">History</span></a></li><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Careers</span></a></li></ul></nav> </div> </div> <div class="wp-block-group is-vertical is-content-justification-stretch is-layout-flex wp-container-core-group-is-layout-d8dce8eb wp-block-group-is-layout-flex"> <h2 class="wp-block-heading has-medium-font-size has-body-font-family" style="font-style:normal;font-weight:600">Privacy</h2> <div class="wp-block-group is-vertical is-layout-flex wp-container-core-group-is-layout-bce0c455 wp-block-group-is-layout-flex"> <nav style="font-style:normal;font-weight:400" class="has-small-font-size is-vertical wp-block-navigation is-layout-flex wp-container-core-navigation-is-layout-bce0c455 wp-block-navigation-is-layout-flex" aria-label="Privacy"><ul style="font-style:normal;font-weight:400" class="wp-block-navigation__container has-small-font-size is-vertical wp-block-navigation" aria-label="Privacy"><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Privacy Policy</span></a></li><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Terms and Conditions</span></a></li><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Contact Us</span></a></li></ul></nav> </div> </div> <div class="wp-block-group is-vertical is-content-justification-stretch is-layout-flex wp-container-core-group-is-layout-d8dce8eb wp-block-group-is-layout-flex"> <h2 class="wp-block-heading has-medium-font-size has-body-font-family" style="font-style:normal;font-weight:600">Social</h2> <div class="wp-block-group is-vertical is-layout-flex wp-container-core-group-is-layout-bce0c455 wp-block-group-is-layout-flex"> <nav style="font-style:normal;font-weight:400" class="has-small-font-size is-vertical wp-block-navigation is-layout-flex wp-container-core-navigation-is-layout-bce0c455 wp-block-navigation-is-layout-flex" aria-label="Social Media"><ul style="font-style:normal;font-weight:400" class="wp-block-navigation__container has-small-font-size is-vertical wp-block-navigation" aria-label="Social Media"><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Facebook</span></a></li><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Instagram</span></a></li><li class="wp-block-navigation-item wp-block-navigation-link"><a class="wp-block-navigation-item__content" href="#"><span class="wp-block-navigation-item__label">Twitter/X</span></a></li></ul></nav> </div> </div> </div> </div> </div> <div class="wp-block-group alignwide is-layout-flow wp-block-group-is-layout-flow" style="padding-top:var(--wp--preset--spacing--50);padding-bottom:0"> <p class="has-contrast-2-color has-text-color has-link-color has-small-font-size wp-elements-1 wp-block-paragraph"> Designed with <a href="https://wordpress.org" rel="nofollow">WordPress</a> </p> </div> </div> </footer> </div> <script type="speculationrules"> {"prefetch":[{"source":"document","where":{"and":[{"href_matches":"/*"},{"not":{"href_matches":["/wp-*.php","/wp-admin/*","/wp-content/uploads/*","/wp-content/*","/wp-content/plugins/*","/wp-content/themes/twentytwentyfour/*","/*\\?(.+)"]}},{"not":{"selector_matches":"a[rel~=\"nofollow\"]"}},{"not":{"selector_matches":".no-prefetch, .no-prefetch a"}}]},"eagerness":"conservative"}]} </script> <script data-wp-router-options="{"loadOnClientNavigation":true}" fetchpriority="low" id="@wordpress/block-library/navigation/view-js-module" src="https://ziliao.tiaomu.com/wp-includes/js/dist/script-modules/block-library/navigation/view.min.js?ver=1bf28ded04f9f188bdcb" type="module"></script> <script id="wp-emoji-settings" type="application/json"> {"baseUrl":"https://s.w.org/images/core/emoji/17.0.2/72x72/","ext":".png","svgUrl":"https://s.w.org/images/core/emoji/17.0.2/svg/","svgExt":".svg","source":{"concatemoji":"https://ziliao.tiaomu.com/wp-includes/js/wp-emoji-release.min.js?ver=7.1.3"}} </script> <script type="module"> /*! This file is auto-generated */ var e="script#wp-emoji-settings",t=document.querySelector(e);if(!(t instanceof HTMLScriptElement))throw new Error("Element missing: "+e);const r=JSON.parse(t.text),s=(window._wpemojiSettings=r,"wpEmojiSettingsSupports"),o=["flag","emoji"];function i(e){try{var t={supportTests:e,timestamp:(new Date).valueOf()};sessionStorage.setItem(s,JSON.stringify(t))}catch(e){}}function c(e,t,n){e.clearRect(0,0,e.canvas.width,e.canvas.height),e.fillText(t,0,0);t=new Uint32Array(e.getImageData(0,0,e.canvas.width,e.canvas.height).data);e.clearRect(0,0,e.canvas.width,e.canvas.height),e.fillText(n,0,0);const r=new Uint32Array(e.getImageData(0,0,e.canvas.width,e.canvas.height).data);return t.every((e,t)=>e===r[t])}function p(e,t){e.clearRect(0,0,e.canvas.width,e.canvas.height),e.fillText(t,0,0);var n=e.getImageData(16,16,1,1);for(let e=0;e<n.data.length;e++)if(0!==n.data[e])return!1;return!0}function u(e,t,n,r){switch(t){case"flag":return n(e,"\ud83c\udff3\ufe0f\u200d\u26a7\ufe0f","\ud83c\udff3\ufe0f\u200b\u26a7\ufe0f")?!1:!n(e,"\ud83c\udde8\ud83c\uddf6","\ud83c\udde8\u200b\ud83c\uddf6")&&!n(e,"\ud83c\udff4\udb40\udc67\udb40\udc62\udb40\udc65\udb40\udc6e\udb40\udc67\udb40\udc7f","\ud83c\udff4\u200b\udb40\udc67\u200b\udb40\udc62\u200b\udb40\udc65\u200b\udb40\udc6e\u200b\udb40\udc67\u200b\udb40\udc7f");case"emoji":return!r(e,"\ud83e\u1fac8")}return!1}function f(e,t,n,r){let a;const s=(a="undefined"!=typeof WorkerGlobalScope&&self instanceof WorkerGlobalScope?new OffscreenCanvas(300,150):document.createElement("canvas")).getContext("2d",{willReadFrequently:!0}),o=(s.textBaseline="top",s.font="600 32px Arial",{});return e.forEach(e=>{o[e]=t(s,e,n,r)}),o}function a(e){var t=document.createElement("script");t.src=e,t.defer=!0,document.head.appendChild(t)}r.supports={everything:!0,everythingExceptFlag:!0},new Promise(t=>{let n=function(){try{var e=JSON.parse(sessionStorage.getItem(s));if("object"==typeof e&&"number"==typeof e.timestamp&&(new Date).valueOf()<e.timestamp+604800&&"object"==typeof e.supportTests)return e.supportTests}catch(e){}return null}();if(!n){if("undefined"!=typeof Worker&&"undefined"!=typeof OffscreenCanvas&&"undefined"!=typeof URL&&URL.createObjectURL&&"undefined"!=typeof Blob)try{var e="postMessage("+f.toString()+"("+[JSON.stringify(o),u.toString(),c.toString(),p.toString()].join(",")+"));",r=new Blob([e],{type:"text/javascript"});const a=new Worker(URL.createObjectURL(r),{name:"wpTestEmojiSupports"});return void(a.onmessage=e=>{i(n=e.data),a.terminate(),t(n)})}catch(e){}i(n=f(o,u,c,p))}t(n)}).then(e=>{for(const n in e)r.supports[n]=e[n],r.supports.everything=r.supports.everything&&r.supports[n],"flag"!==n&&(r.supports.everythingExceptFlag=r.supports.everythingExceptFlag&&r.supports[n]);var t;r.supports.everythingExceptFlag=r.supports.everythingExceptFlag&&!r.supports.flag,r.supports.everything||((t=r.source||{}).concatemoji?a(t.concatemoji):t.wpemoji&&t.twemoji&&(a(t.twemoji),a(t.wpemoji)))}); //# sourceURL=https://ziliao.tiaomu.com/wp-includes/js/wp-emoji-loader.min.js </script> <noscript><link rel="stylesheet" href="https://ziliao.tiaomu.com/wp-content/themes/abiz/assets/css/all.min.css?ver=6.5.1"></noscript> </body> </html>