数据分析 on 进击的骑士 https://knight134.com/tags/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/ Recent content in 数据分析 on 进击的骑士 Hugo zh-cn Fri, 17 May 2024 16:00:00 +0800 SQL 查询优化：EXPLAIN 命令的妙用 https://knight134.com/posts/2024-05-17-sql-optimization/ Fri, 17 May 2024 16:00:00 +0800 https://knight134.com/posts/2024-05-17-sql-optimization/ <p>当数据量达到百万级时，一条慢查询足以拖垮整个服务。学会使用 <code>EXPLAIN</code> 来分析 SQL 执行计划是必备技能。 主要关注 <code>type</code> 列，如果是 <code>ALL</code> 表示全表扫描，需要优化索引。</p> 使用 Pandas 进行数据清洗实战 https://knight134.com/posts/2024-05-12-data-analysis-pandas/ Sun, 12 May 2024 10:30:00 +0800 https://knight134.com/posts/2024-05-12-data-analysis-pandas/ <p>在数据分析的工作中，80% 的时间都在做数据清洗。今天记录一下使用 Pandas 处理缺失值和异常值的常用技巧。</p> <h3 id="处理缺失值">处理缺失值</h3> <ul> <li><code>dropna()</code>：直接丢弃含有缺失值的行。</li> <li><code>fillna()</code>：用均值、中位数或特定值填充。</li> </ul> <h3 id="数据透视表">数据透视表</h3> <p>Pandas 的 <code>pivot_table</code> 类似于 Excel 的数据透视表，非常强大。 python</p> <p>import pandas as pd</p> <p>df = pd.read_csv(&lsquo;data.csv&rsquo;)</p> <p>table = pd.pivot_table(df, values=&lsquo;sales&rsquo;, index=&lsquo;region&rsquo;, columns=&lsquo;year&rsquo;, aggfunc=&lsquo;sum&rsquo;)</p> <p>print(table)</p> 我的技术探索之旅：从数据分析到前端可视化 https://knight134.com/posts/my-first-post/ Mon, 15 Jan 2024 10:00:00 +0800 https://knight134.com/posts/my-first-post/ <p>这里是你的文章内容。你可以愉快地使用 <strong>Markdown</strong> 语法。</p> <h3 id="我的第一个数据可视化项目">我的第一个数据可视化项目</h3> <p>比如，你可以直接把你在 Jupyter Notebook 里生成的前端可视化代码（HTML/JS）直接粘贴在这里：</p> <div id="my-chart" style="width: 600px; height: 400px; background: #f0f0f0; display: flex; align-items: center; justify-content: center;"> <!------ <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>D3.js Solar System</title> <script src="https://d3js.org/d3.v7.min.js"></script> <style> body { margin: 0; padding: 0; display: flex; justify-content: center; align-items: center; min-height: 100vh; background-color: #000; overflow: hidden; /* 防止滚动条出现 */ } .solar-system { width: 800px; height: 800px; } /* 星空背景 */ .stars { fill: white; opacity: 0.8; } /* 行星公转动画 */ @keyframes orbit { from { transform: rotate(0deg); } to { transform: rotate(360deg); } } .orbit { animation: orbit linear infinite; transform-origin: center center; } /* 行星自转动画 */ @keyframes spin { from { transform: rotate(0deg); } to { transform: rotate(-360deg); } } .planet-spin { animation: spin linear infinite; transform-origin: center center; } </style> </head> <body> <pre><code>&lt;div id=&quot;solar-system-container&quot;&gt;&lt;/div&gt; &lt;script&gt; // 1. 定义太阳系数据 const solarSystem = { sun: { radius: 40, color: &quot;#FDB813&quot; }, planets: [ { name: &quot;水星&quot;, radius: 8, color: &quot;#A9A9A9&quot;, distance: 100, orbitDuration: 4, spinDuration: 10 }, { name: &quot;金星&quot;, radius: 12, color: &quot;#FFC649&quot;, distance: 160, orbitDuration: 8, spinDuration: 20 }, { name: &quot;地球&quot;, radius: 14, color: &quot;#6B93D6&quot;, distance: 220, orbitDuration: 12, spinDuration: 5 }, { name: &quot;火星&quot;, radius: 10, color: &quot;#FF4D4D&quot;, distance: 280, orbitDuration: 16, spinDuration: 15 }, { name: &quot;木星&quot;, radius: 30, color: &quot;#FFA726&quot;, distance: 380, orbitDuration: 24, spinDuration: 3 }, { name: &quot;土星&quot;, radius: 26, color: &quot;#FFD95A&quot;, distance: 480, orbitDuration: 30, spinDuration: 4, hasRing: true }, { name: &quot;天王星&quot;, radius: 20, color: &quot;#4FD0E7&quot;, distance: 580, orbitDuration: 40, spinDuration: 8 }, { name: &quot;海王星&quot;, radius: 20, color: &quot;#4B70DD&quot;, distance: 680, orbitDuration: 50, spinDuration: 10 } ] }; // 2. 创建 SVG 画布 const width = 800; const height = 800; const centerX = width / 2; const centerY = height / 2; const svg = d3.select(&quot;#solar-system-container&quot;) .append(&quot;svg&quot;) .attr(&quot;width&quot;, width) .attr(&quot;height&quot;, height) .attr(&quot;viewBox&quot;, `0 0 ${width} ${height}`) .attr(&quot;preserveAspectRatio&quot;, &quot;xMidYMid meet&quot;) .classed(&quot;solar-system&quot;, true); // 3. 绘制背景星空 const starsCount = 200; const starsData = Array.from({ length: starsCount }, () =&gt; ({ x: Math.random() * width, y: Math.random() * height, r: Math.random() * 1.5 })); svg.selectAll(&quot;.star&quot;) .data(starsData) .enter() .append(&quot;circle&quot;) .attr(&quot;class&quot;, &quot;stars&quot;) .attr(&quot;cx&quot;, d =&gt; d.x) .attr(&quot;cy&quot;, d =&gt; d.y) .attr(&quot;r&quot;, d =&gt; d.r); // 4. 绘制太阳 svg.append(&quot;circle&quot;) .attr(&quot;cx&quot;, centerX) .attr(&quot;cy&quot;, centerY) .attr(&quot;r&quot;, solarSystem.sun.radius) .attr(&quot;fill&quot;, solarSystem.sun.color) .attr(&quot;filter&quot;, &quot;url(#glow)&quot;); // 添加发光滤镜 // 5. 绘制行星及其轨道 const planetGroups = svg.selectAll(&quot;.planet-group&quot;) .data(solarSystem.planets) .enter() .append(&quot;g&quot;) .attr(&quot;class&quot;, &quot;planet-group&quot;) .attr(&quot;transform&quot;, `translate(${centerX}, ${centerY})`); // 轨道线 planetGroups.append(&quot;circle&quot;) .attr(&quot;class&quot;, &quot;orbit&quot;) .attr(&quot;r&quot;, d =&gt; d.distance) .attr(&quot;fill&quot;, &quot;none&quot;) .attr(&quot;stroke&quot;, &quot;rgba(255, 255, 255, 0.2)&quot;) .attr(&quot;stroke-width&quot;, 1) .style(&quot;animation-duration&quot;, d =&gt; `${d.orbitDuration}s`); // 行星本体 const planets = planetGroups.append(&quot;g&quot;) .attr(&quot;class&quot;, &quot;planet-spin&quot;) .style(&quot;animation-duration&quot;, d =&gt; `${d.spinDuration}s`); planets.append(&quot;circle&quot;) .attr(&quot;cx&quot;, d =&gt; d.distance) .attr(&quot;cy&quot;, 0) .attr(&quot;r&quot;, d =&gt; d.radius) .attr(&quot;fill&quot;, d =&gt; d.color); // 土星环 planets.filter(d =&gt; d.hasRing) .append(&quot;ellipse&quot;) .attr(&quot;cx&quot;, d =&gt; d.distance) .attr(&quot;cy&quot;, 0) .attr(&quot;rx&quot;, d =&gt; d.radius * 2.5) .attr(&quot;ry&quot;, d =&gt; d.radius * 0.5) .attr(&quot;fill&quot;, &quot;none&quot;) .attr(&quot;stroke&quot;, d =&gt; d3.color(d.color).brighter(0.5)) .attr(&quot;stroke-width&quot;, 3) .attr(&quot;opacity&quot;, 0.8) .attr(&quot;transform&quot;, `rotate(-20, ${d =&gt; d.distance}, 0)`); // 6. 添加太阳光晕滤镜 const defs = svg.append(&quot;defs&quot;); const filter = defs.append(&quot;filter&quot;) .attr(&quot;id&quot;, &quot;glow&quot;) .attr(&quot;x&quot;, &quot;-50%&quot;) .attr(&quot;y&quot;, &quot;-50%&quot;) .attr(&quot;width&quot;, &quot;200%&quot;) .attr(&quot;height&quot;, &quot;200%&quot;); filter.append(&quot;feGaussianBlur&quot;) .attr(&quot;stdDeviation&quot;, &quot;8&quot;) .attr(&quot;result&quot;, &quot;coloredBlur&quot;); const feMerge = filter.append(&quot;feMerge&quot;); feMerge.append(&quot;feMergeNode&quot;).attr(&quot;in&quot;, &quot;coloredBlur&quot;); feMerge.append(&quot;feMergeNode&quot;).attr(&quot;in&quot;, &quot;SourceGraphic&quot;); &lt;/script&gt; </code></pre> </body> </html> ------> <!-- 1. 样式部分：直接写在这里，或者放在文章的 <head> 里 --> <!----------------- <style> /* 星空背景，覆盖父容器的灰色背景 */ #my-chart { background-color: #000 !important; /* 强制变黑，配合太空主题 */ position: relative; overflow: hidden; } /* 星空点点 */ .stars { fill: white; opacity: 0.8; } /* 行星公转动画 */ @keyframes orbit { from { transform: rotate(0deg); } to { transform: rotate(360deg); } } .orbit { animation: orbit linear infinite; transform-origin: center center; } /* 行星自转动画 */ @keyframes spin { from { transform: rotate(0deg); } to { transform: rotate(-360deg); } } .planet-spin { animation: spin linear infinite; transform-origin: center center; } </style> --------------> <!-- 2. 内容容器（保留你的原ID，但清空内容） --> <p>/*</p>