包含标签 mongodb articles

Crawl a website with scrapy

Introduction In this article, we are going to see how to scrape information from a website, in particular, from all pages with a common URL pattern. We will see how to do that with Scrapy, a very powerful, and yet simple, scraping and web-crawling framework. For example, you might be interested in scraping information about each article of a blog, and store it information in a database. To achieve such a thing, we will see how to implement a simple spider using Scrapy, which will crawl the blog and store the extracted data into a MongoDB database. We will consider that you have a working MongoDB server, and that you have installed thepymongoandscrapypython packages, both installable with pip. If you have never toyed around with Scrapy, you should first read this short tutorial. First step, identify the URL pattern(s) In……

Continue reading

PHP添加MongoDB扩展

由于要使用mikoomi mongodb plugin插件,所以需要php对mongodb的扩展支持,默认通过源安装的php并没有mongodb的扩展支持,具体可以通过php -m|grep mongo 验证 。这里就结合phpize提供的动态扩展功能进行mongodb的扩展。 一、php mongo模块的编译 先从 http://pecl.php.net/package/mongo 下载目前的……

Continue reading

mongodb备份与恢复

mongodb是目前最流行的nosql数据库,其自身也提供了备份与恢复命令 。具体程序为mongodump和mongorestore 。 一、mongodump备份 mongodump的具体用法可以查看帮助: 1[root@web20 ~]# /App/mongodb/bin/mongodump -h 2ERROR: required parameter is missing in 'host' 3Export MongoDB data to BSON files. 4options: 5 --help produce help message 6 -v [ --verbose ] be more verbose (include multiple times for more 7 verbosity e.g. -vvvvv)……

Continue reading

mongodb的慢查询

在MySQL中,慢查询日志是经常作为我们优化数据库的依据,那在MongoDB中是否有类似的功能呢?答案是肯定的,那就是Mongo Database Profiler.不仅有,而且还有一些比MySQL的Slow Query Log更详细的信息。它就是我们这篇文章的主题。 开启 Profiling 功能 有两种方式可以控制 Profiling 的开关和级别……

Continue reading

mongodb强制一个成员为主

公司有一部分安全性要求不是十分高的数据使用的mongodb,其查询和插入性能确实较ACID理论的数据库来说要好的多。不过数据安全性上确实不如ACID型。为了补冲这点不足,做了replica sets(复制)。但由于操作时,没有进行优先级设置。所以想将某台性能略好的机器设置为主。 最早……

Continue reading

mongo内存管理

但凡初次接触MongoDB的人,无不惊讶于它对内存的贪得无厌,至于个中缘由,我先讲讲Linux是如何管理内存的,再说说MongoDB是如何使用内存的,答案自然就清楚了。 据说带着问题学习更有效,那就先看一个MongoDB服务器的top命令结果: 1shell> top -p $(pidof mongod) 2Mem: 32872124k total, 30065320k used, 2806804k free, 245020k buffers 3Swap: 2097144k total, 100k used,……

Continue reading

mongodb与mysql命令对比

传统的关系数据库一般由数据库(database)、表(table)、记录(record)三个层次概念组成,MongoDB是由数据库(database)、集合(collection)、文档对象(document) 三个层次组成。MongoDB对于关系型数据库里的表,但是集合中没有列、……

Continue reading

mongo与关系型数据库的使用对比

一、执行程序的区别 MySQL executable Oracle executable Mongo executable mysqld oracle mongod mysql sqlplus mongo 二、库、表、行、列等概念的区别 MySQL term Mongo term/concept database database table collection index index row BSON document column BSON field join embedding and linking primary key _id field group by aggregation 三、语句用法上的区别 SQL Statement Mongo Statement CREATE TABLE USERS (a Number, b Number) db.createCollection(“mycoll”) ALTER TABLE users ADD … implicit INSERT INTO USERS VALUES(3,5) db.users.insert({a:3,b:5}) SELECT a,b FROM users db.users.find({}, {a:1,b:1}) SELECT * FROM users db.users.find() SELECT * FROM users WHERE……

Continue reading

mongodb常用命令

一、用户操作命令: 1#增加或修改用户密码 2db.addUser('admin','pwd') 3#查看用户列表 4db.system.users.find() 5#用户认证 6db.auth('admin','pwd') 7#删除用户 8db.removeUser('mongodb') 9#查看所有用户 10show users 11#查看所有数据库 12show dbs 13#查看所有的collection 14show collections 15#查看各collection的状态 16db.printCollectionStats() 17#查看主从复制状态 18db.printReplicationInfo() 19#修复数据库 20db.repairDatabase() 21#设置记录profil……

Continue reading

Latest articles

Categories

Tags

ACL AD AES AI AWS Ansible Atlassian Azure BMC Blockchain Brocade CDH5 CL210 Cobbler Confd C语言 DDOS DISTINCT DNS EKS ELK GCP Ghost Git Glusterfs Go Godaddy Grafana HBA HCIE Hotspot HttpWatch IBM IIS IOS InfluxDB Ingress InnoDB JavaScript Jinja2 KVM Keepalived Mplayer MySQLdb Netlify OpenResty PM PostgreSQL QoS RH318 RH442 RHCA RHCE RHEV RSA SRE SecureCRT Statuscode SublimeText2 TC Telecom Tencentcloud VBA aira2 alpine android anpic apache apm apparmor appfog apr apt-get aria2 array atop audit awk awstats axel backdoor backup bamboo bash bat benchmark bigdata bin bind bitwise book bootstrap bsd c1000 cache capistrano catlog centos centos7 chatops chattr check_mk checkinstall cisco clearall clickhouse cloud-desktop cmdb cms collectd comm compress conver corosync cpu crontab crunchbang css curl date decode dell desktop devops df dhcp diff diskpart django docker dos2unix dpkg drupal etcd excel fail2ban fastcgi fdisk fiddler find firewalld flask flvtool ftp function fuser geek gin github gitlab glances golang google gooupadd graphviz gravatra grep grub2 hadoop haproxy hardware heartbeat helm hexdump hhvm history html http/html/web httplogs https huawei huaweicloud hugo icmp iconv ifconfig inotify iopp ipmitool iptables iredmail iscsi isito it-news java jdk jenkins jira join joomla k3s k8s kdump kernel kingate lamp last leetcode lib light-http linux linux高级篇 ln ls lsi lsof lvm lvs mac mail man mark markdown matplotlib maven memcached microservice mimikatz mkdocs mkpasswd mmonit mod_jk mongodb monit monitor mono moodle mosh mount mpm mrtg mtr my.cnf mysql mysqlbinlog mysqld_multi mysqldump mysqlhotcopy nagios nc nethogs nexus nfs nginx nmon nocatlog node.js nrpe ntfs ntop ntp obs ocr open-falcon openbox opencv openldap openssl openstack oracle oswatch paas pacemaker pam pandas parted pcp pcre pdf percona perl pexpect pgrep php php-fpm ping plsql develope postfix powershell prettify proc prometheus puppeteer pushd pwgen pxe pyecharts python python模块 radmin raid rdesktop read redhat redis redmine regex rh134 rhel7 rhel8 rm rman rootkit route rpm rpmforge rrdtool rsync rsyslog safe saltstack samba scapy screen sed selenium selinux seo seq session set shc sheepdog shell shopt sitemap skydns smokeping snffier snmp socket soft sort spider sql sqlserver squid ss ssh sshpass strace strings su sudo suse svn sysbench syslog-ng sysstat systemd t tar tcpcopy tcpdump tech telnet tengine test testlink threads time tmux tomcat touch tr tsar twisted ubuntu udev ulimit unix unixbench user-agent useradd varnish vbs vercel vi vim visudo vmstat vmware vnc voice vpn vscode vsftp vsftpd vue watchdog web webcam webistrano wget wiki windows wol wordpress workshop wsl x-windows xampp xcache xmllint xtrabackup yule yum zabbix zeromq zip zonetime zookeeper 下载工具 云主机 云原生 代理 加密 古意 吐槽 圈里圈外 娱乐 字符串函数 安全 平台架构 意林 推理 提权 故事汇 故障案例 数据结构 每日看点 民国史 生活 科学记录 站长管理工具 算法 管理 网站架构 翻墙 股票 行业 诗韵 负载均衡 远程管理 面试题

Links

Meta